Tits In Tops Thread Scraper Tool / How to download TiTs pictures?

mason2371

mason2371

Active Member
Joined
Sep 22, 2019
Messages
84
Reaction score
181
Hey all,

Over the past few days, I've developed a tool to scrape threads from this site, and I figured I'd share it here. Currently, it's just a python script; if anyone would like to contribute with a GUI or Windows installer, I'm sure some here would appreciate it.

Features:
  • Download all media posted to a thread on this site (images, videos, and attached files). Should work for full-size images/videos, thumbnails, and attachments
  • Save any URLs found posted in the thread to a file
  • Record downloaded URLs to a file, so subsequent runs will not re-download files
  • If multiple files are posted with the same name, they will be renamed to not conflict with each other (e.g. 'image1 (1).jpg', 'image1 (2).jpg')
Issues/Limitations:
  • Requires titsintops username and password as command-line arguments
  • Does not check for duplicate files, so if the same file is posted to the thread twice, you'll get that image both times
  • No GUI
  • No installer, so providing python requirements is up to the user
  • I didn't test it thoroughly, especially against user error. However, I've archived a few threads already and haven't noticed any issues (even with threads with >500 files).
If anyone has questions, let me know.
 

Attachments

  • tit-thread-scraper.zip
    5 KB · Views: 460
Wow, thanks! I was just going to make one and your topic appeared, wwhat a coincidence!
 
Is there a way to automatically download all of the full size picks from the posts here so they can be viewed all at once without having to keep opening and closing things or saving them one by one? I normally use Bulk Image Downloader or JDownloader2, but they seem not to work with this forum.
 
Last edited:
cocopopper000 said:
Is there a way to automatically download all of the full size picks from the posts here so they can be viewed all at once without having to keep opening and closing things or saving them one by one? I normally use Bulk Image Downloader or JDownloader2, but they seem not to work with this forum.
Click to expand...
Not that I'm aware of.
Of course, I've never really tried.

Perhaps someone has it figure out?
 
It came out of a bookmarklet I made to open all images in new tabs for easier saving. If you're not comfortable with a Python cli tool I can reverse-engineer the bookmarklet again.

Edit: The bookmark I made is posted here

Update:
Been a while, but I think all that's different in this version is a fix for weird filenames and properly url decoding them.

Also, an example (in Bash on Linux):
Bash: 
cd '/path/to/porn/folder/Milena Velba' && \
mkdir -p tit && \
~/Downloads/tit-thread-scraper.py \
    --username 'tit-username@example.com' \
    --password 'tit-password' \
    --links 'urls.unsorted' \
    --archive 'archive.tit.txt' \
    'https://titsintops.com/phpBB2/index.php?threads/milena-velba-appreciation-thread-no-full-photo-sets.1539/'

Notes:
  • By now, this script has been pretty well tested and seems to work great. Off the Milena Velba thread, I've downloaded 14GB, with 25556 total files.
  • The script might infinite loop when the provided thread url has a /page-xx suffix, or #post-xxxxxx prefix, so make sure to remove those yourself since I'm too lazy to strip them in the code
  • I know that providing a password on the command line isn't secure, but you shouldn't be running this on the public library's computers anyway
If anyone wants help using or contributing, dm me
 

Attachments

  • tit-thread-scraper.zip
    5.3 KB · Views: 153
Last edited:
This thread deserves to be more highlighted just for the concept. This is mandatorily useful to improve the experience on this site.

Judging by the OP, this works like RipMe, right?
 
Last edited:
A9570 said:
This thread deserves to be more highlighted just for the concept. This is mandatorily useful to improve the experience on this site.

Judging by the OP, this works like RipMe, right?
Click to expand...
Pretty much, though my direct inspiration was gallery-dl and youtube-dl.
 
Code: 
python main.py -u 'AmanteLateral' -p 'mypassword' -l 'urls.unsorted' -a 'archive.tit.txt' 'https://titsintops.com/phpBB2/index.php?threads/isabellar-isabella-r-big-naturals-from-venezuela.103261/'

For some reason this doesn't work on Windows's CMD (not even as admin). It doesn't show any errors it simply closes right after I run it and doesn't download a single file.

I had to use window's git on bash to make it work.
 
AmanteLateral said:
Code: 
python main.py -u 'AmanteLateral' -p 'mypassword' -l 'urls.unsorted' -a 'archive.tit.txt' 'https://titsintops.com/phpBB2/index.php?threads/isabellar-isabella-r-big-naturals-from-venezuela.103261/'

For some reason this doesn't work on Windows's CMD (not even as admin). It doesn't show any errors it simply closes right after I run it and doesn't download a single file.

I had to use window's git on bash to make it work.
Click to expand...
Strange that it works on bash. Maybe it's a quoting issue; cmd might only accept double quotes.
 
mason2371 said:
Strange that it works on bash. Maybe it's a quoting issue; cmd might only accept double quotes.
Click to expand...
Same thing happens with double quotes.
I hope it's only something weird on my end but would be nice if anyone can confirm if they have the same issue.
 

Attachments

  • Capture.PNG
    Capture.PNG
    13.9 KB · Views: 127
I am getting errors on mac

SyntaxError: invalid syntax


python ./tit-thread-scraper.py


File "./tit-thread-scraper.py", line 28


def __init__(self, db: Optional[Iterator[str]] = None):


^
 
@AmanteLateral and @mplscpl can you both run
Bash: 
python -VV
in your terminals and post what it prints?
 
mplscpl said:
Python 2.7.16
Click to expand...
That's what I thought. You'll need to upgrade to Python 3. That should fix it.

@AmanteLateral You're on the same version I built it on, so that's definitely not the problem. Maybe try running in cmd:
Bash: 
python main.py --help
If that prints the help message, then it's probably some kind of weirdness with the parameters caused by cmd. You can also try
Bash: 
python -i main.py --help
which will keep you in interactive python when the script finishes. If you aren't kept in python but are kicked back out to regular cmd then we know that it's not even running the script.
 
mason2371 said:
That's what I thought. You'll need to upgrade to Python 3. That should fix it.

@AmanteLateral You're on the same version I built it on, so that's definitely not the problem. Maybe try running in cmd:
Bash: 
python main.py --help
If that prints the help message, then it's probably some kind of weirdness with the parameters caused by cmd. You can also try
Bash: 
python -i main.py --help
which will keep you in interactive python when the script finishes. If you aren't kept in python but are kicked back out to regular cmd then we know that it's not even running the script.
Click to expand...

Here's the output for both commands


Code: 
D:\Software\tit-thread-scraper>python main.py --help
usage: main.py [-h] [-o OUTPUT_DIRECTORY] [-n] -u USERNAME -p PASSWORD [-v] [-q] [-c] [-l LINKS] [-a ARCHIVE]
               thread_url

positional arguments:
  thread_url            URL of the thread to download

optional arguments:
  -h, --help            show this help message and exit
  -o OUTPUT_DIRECTORY, --output-directory OUTPUT_DIRECTORY
                        Directory to download media files to. Default is '.'
  -n, --dry-run         Do not write downloaded media to disk
  -u USERNAME, --username USERNAME
                        titsintops user name
  -p PASSWORD, --password PASSWORD
                        titsintops password
  -v, --verbose         Print more detailed diagnostic messages
  -q, --quiet           Do not print standard output messages. Does not affect --verbose diagnostic messages
  -c, --clobber         Overwrite files that already exist in output directory. Default is to save files with a new
                        name instead.
  -l LINKS, --links LINKS
                        Write links posted in thread to a file
  -a ARCHIVE, --archive ARCHIVE
                        Record downloaded media to a file, and skip media files already listed in the archive

D:\Software\tit-thread-scraper>python -i main.py --help
usage: main.py [-h] [-o OUTPUT_DIRECTORY] [-n] -u USERNAME -p PASSWORD [-v] [-q] [-c] [-l LINKS] [-a ARCHIVE]
               thread_url

positional arguments:
  thread_url            URL of the thread to download

optional arguments:
  -h, --help            show this help message and exit
  -o OUTPUT_DIRECTORY, --output-directory OUTPUT_DIRECTORY
                        Directory to download media files to. Default is '.'
  -n, --dry-run         Do not write downloaded media to disk
  -u USERNAME, --username USERNAME
                        titsintops user name
  -p PASSWORD, --password PASSWORD
                        titsintops password
  -v, --verbose         Print more detailed diagnostic messages
  -q, --quiet           Do not print standard output messages. Does not affect --verbose diagnostic messages
  -c, --clobber         Overwrite files that already exist in output directory. Default is to save files with a new
                        name instead.
  -l LINKS, --links LINKS
                        Write links posted in thread to a file
  -a ARCHIVE, --archive ARCHIVE
                        Record downloaded media to a file, and skip media files already listed in the archive
Traceback (most recent call last):
  File "D:\Software\tit-thread-scraper\main.py", line 474, in <module>
    args = parser.parse_args()
  File "C:\Python39\lib\argparse.py", line 1818, in parse_args
    args, argv = self.parse_known_args(args, namespace)
  File "C:\Python39\lib\argparse.py", line 1851, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)
  File "C:\Python39\lib\argparse.py", line 2060, in _parse_known_args
    start_index = consume_optional(start_index)
  File "C:\Python39\lib\argparse.py", line 2000, in consume_optional
    take_action(action, args, option_string)
  File "C:\Python39\lib\argparse.py", line 1928, in take_action
    action(self, namespace, argument_values, option_string)
  File "C:\Python39\lib\argparse.py", line 1093, in __call__
    parser.exit()
  File "C:\Python39\lib\argparse.py", line 2562, in exit
    _sys.exit(status)
SystemExit: 0
>>>

the second one stays inside the python interpreter
 
AmanteLateral said:
Here's the output for both commands


Code: 
D:\Software\tit-thread-scraper>python main.py --help
usage: main.py [-h] [-o OUTPUT_DIRECTORY] [-n] -u USERNAME -p PASSWORD [-v] [-q] [-c] [-l LINKS] [-a ARCHIVE]
               thread_url

positional arguments:
  thread_url            URL of the thread to download

optional arguments:
  -h, --help            show this help message and exit
  -o OUTPUT_DIRECTORY, --output-directory OUTPUT_DIRECTORY
                        Directory to download media files to. Default is '.'
  -n, --dry-run         Do not write downloaded media to disk
  -u USERNAME, --username USERNAME
                        titsintops user name
  -p PASSWORD, --password PASSWORD
                        titsintops password
  -v, --verbose         Print more detailed diagnostic messages
  -q, --quiet           Do not print standard output messages. Does not affect --verbose diagnostic messages
  -c, --clobber         Overwrite files that already exist in output directory. Default is to save files with a new
                        name instead.
  -l LINKS, --links LINKS
                        Write links posted in thread to a file
  -a ARCHIVE, --archive ARCHIVE
                        Record downloaded media to a file, and skip media files already listed in the archive

D:\Software\tit-thread-scraper>python -i main.py --help
usage: main.py [-h] [-o OUTPUT_DIRECTORY] [-n] -u USERNAME -p PASSWORD [-v] [-q] [-c] [-l LINKS] [-a ARCHIVE]
               thread_url

positional arguments:
  thread_url            URL of the thread to download

optional arguments:
  -h, --help            show this help message and exit
  -o OUTPUT_DIRECTORY, --output-directory OUTPUT_DIRECTORY
                        Directory to download media files to. Default is '.'
  -n, --dry-run         Do not write downloaded media to disk
  -u USERNAME, --username USERNAME
                        titsintops user name
  -p PASSWORD, --password PASSWORD
                        titsintops password
  -v, --verbose         Print more detailed diagnostic messages
  -q, --quiet           Do not print standard output messages. Does not affect --verbose diagnostic messages
  -c, --clobber         Overwrite files that already exist in output directory. Default is to save files with a new
                        name instead.
  -l LINKS, --links LINKS
                        Write links posted in thread to a file
  -a ARCHIVE, --archive ARCHIVE
                        Record downloaded media to a file, and skip media files already listed in the archive
Traceback (most recent call last):
  File "D:\Software\tit-thread-scraper\main.py", line 474, in <module>
    args = parser.parse_args()
  File "C:\Python39\lib\argparse.py", line 1818, in parse_args
    args, argv = self.parse_known_args(args, namespace)
  File "C:\Python39\lib\argparse.py", line 1851, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)
  File "C:\Python39\lib\argparse.py", line 2060, in _parse_known_args
    start_index = consume_optional(start_index)
  File "C:\Python39\lib\argparse.py", line 2000, in consume_optional
    take_action(action, args, option_string)
  File "C:\Python39\lib\argparse.py", line 1928, in take_action
    action(self, namespace, argument_values, option_string)
  File "C:\Python39\lib\argparse.py", line 1093, in __call__
    parser.exit()
  File "C:\Python39\lib\argparse.py", line 2562, in exit
    _sys.exit(status)
SystemExit: 0
>>>

the second one stays inside the python interpreter
Click to expand...
So the script is definitely running and finishing correctly. Try running it without the --archive option and with the --clobber option, and with -vvv for more detailed output. You can also use --dry-run so it won't actually save the files.
 
I did what you said and it worked but then I decided to try my original command with the double quotes again and it works too lol... I didn't delete any of the other files already in the folder (the urls.unsorted and archive txt files) and there were no images on it so I don't know what happened but all's well that ends well, thanks for the help.
 

Attachments

  • weird.png
    weird.png
    12.2 KB · Views: 77
works great, thanks for developing this
 
You must log in or register to reply here.

Similar threads

P
Discussing the Tits In Tops Hall Of Fame thread
Replies
16
Views
2,039
Nevermore
Nevermore
P
Lima Beans: Super Busty Facebook Reels of Tits In Tops
Replies
1
Views
898
Tscott7
T
Zweii
Can't reach Tits in Tops with Brave browser
Replies
1
Views
257
Alacres
A
blabba75
Boob Curtains / Loose crop tops draped over big tits
Replies
3
Views
792
bknewbie
B
8
Metallic tits in tops
Replies
1
Views
414
Chiuaua223
Chiuaua223
Top