TikTok Scraper – Tool to Download Video Posts, Collect User/trend/hashtag/music feed metadata, sign URL.

TikTok Scraper Logo - Tool to Download Tiktok Video Posts, Collect User, trends, hashtag, music, TikTok feed, metadata, sign URL

TikTok is one of biggest social media that contains billions of user arround the world. This is huge platform to advertise and reach global people. To analyze TikTok users or informations you need a data on specific issues. But how to scrape data from TikTok, to answer this question you can do it manually or you can use some tools to extract data from TikTok.

TikTok Scraper and Downloader

Scrape and download useful information from TikTok.

No login or password are required

This is not an official API support and etc. This is just a scraper that is using TikTok Web API to scrape media and related meta information.

Features

  • Download unlimited post metadata from the User, Hashtag, Trends, or Music-Id pages
  • Save post metadata to the JSON/CSV files
  • Download media with and without the watermark and save to the ZIP file
  • Download single video without the watermark from the CLI
  • Sign URL to make custom request to the TikTok API
  • Extract metadata from the User, Hashtag and Single Video pages
  • Save previous progress and download only new videos that weren’t downloaded before. This feature only works from the CLI and only if download flag is on.
  • View and manage previously downloaded posts history in the CLI
  • Scrape and download user, hashtag, music feeds and single videos specified in the file in batch mode

To Do

  •  CLI: save progress to avoid downloading same videos
  •  Rewrite everything in TypeScript
  •  Improve proxy support
  •  Add tests
  •  Download video without the watermark
  •  Indicate in the output file(csv/json) if the video was downloaded or not
  •  Build and run from Docker
  •  CLI: Scrape and download in batch
  •  CLI: Load proxies from a file
  •  CLI: Optional ZIP
  •  Renew API
  •  Set WebHook URL (CLI)
  •  Add new method to collect music metadata

Installation

tiktok-scraper requires Node.js v10+ to run.

Install from NPM

npm i -g tiktok-scraper

Install from YARN

yarn global add tiktok-scraper

Usage

$ tiktok-scraper --help

Usage: tiktok-scraper <command> [options]

Commands:
  tiktok-scraper user [id]     Scrape videos from username. Enter only username
  tiktok-scraper hashtag [id]  Scrape videos from hashtag. Enter hashtag without #
  tiktok-scraper trend         Scrape posts from current trends
  tiktok-scraper music [id]    Scrape posts from a music id number
  tiktok-scraper video [id]    Download single video without the watermark
  tiktok-scraper history       View previous download history
  tiktok-scraper from-file [file] [async]  Scrape users, hashtags, music, videos mentioned
                                in a file. 1 value per 1 line

Options:
  --version            Show version number                             [boolean]
  --session            Set session cookie value. Sometimes session can be
                       helpful when scraping data from any method  [default: ""]
  --session-file       Set path to the file with list of active sessions. One
                       session per line!                           [default: ""]
  --timeout            Set timeout between requests. Timeout is in Milliseconds:
                       1000 mls = 1 s                               [default: 0]
  --number, -n         Number of posts to scrape. If you will set 0 then all
                       posts will be scraped                        [default: 0]
  --since              Scrape no posts published before this date (timestamp).
                       If set to 0 the filter is deactived          [default: 0]
  --proxy, -p          Set single proxy                            [default: ""]
  --proxy-file         Use proxies from a file. Scraper will use random proxies
                       from the file per each request. 1 line 1 proxy.
                                                                   [default: ""]
  --download, -d       Download video posts to the folder with the name input
                       [id]                           [boolean] [default: false]
  --asyncDownload, -a  Number of concurrent downloads               [default: 5]
  --hd                 Download video in HD. Video size will be x5-x10 times
                       larger and this will affect scraper execution speed. This
                       option only works in combination with -w flag
                                                      [boolean] [default: false]
  --zip, -z            ZIP all downloaded video posts [boolean] [default: false]
  --filepath           File path to save all output files.
      [default: "/Users/karl.wint/Documents/projects/javascript/tiktok-scraper"]
  --filetype, -t       Type of the output file where post information will be
                       saved. 'all' - save information about all posts to the`
                       'json' and 'csv'
                               [choices: "csv", "json", "all", ""] [default: ""]
  --filename, -f       Set custom filename for the output files    [default: ""]
  --noWaterMark, -w    Download video without the watermark. NOTE: With the
                       recent update you only need to use this option if you are
                       scraping Hashtag Feed. User/Trend/Music feeds will have
                       this url by default            [boolean] [default: false]
  --store, -s          Scraper will save the progress in the OS TMP or Custom
                       folder and in the future usage will only download new
                       videos avoiding duplicates     [boolean] [default: false]
  --historypath        Set custom path where history file/files will be stored
                   [default: "/var/folders/d5/fyh1_f2926q7c65g7skc0qh80000gn/T"]
  --remove, -r         Delete the history record by entering "TYPE:INPUT" or
                       "all" to clean all the history. For example: user:bob
                                                                   [default: ""]
  --webHookUrl         Set webhook url to receive scraper result as HTTP
                       requests. For example to your own API       [default: ""]
  --method             Receive data to your webhook url as POST or GET request
                                      [choices: "GET", "POST"] [default: "POST"]
  --help               Show help                                       [boolean]

Examples:
  tiktok-scraper user USERNAME -d -n 100 --session sid_tt=dae32131231
  tiktok-scraper trend -d -n 100 --session sid_tt=dae32131231
  tiktok-scraper hashtag HASHTAG_NAME -d -n 100 --session sid_tt=dae32131231
  tiktok-scraper music MUSIC_ID -d -n 50 --session sid_tt=dae32131231
  tiktok-scraper video https://www.tiktok.com/@tiktok/video/6807491984882765062 -d
  tiktok-scraper history
  tiktok-scraper history -r user:bob
  tiktok-scraper history -r all
  tiktok-scraper from-file BATCH_FILE ASYNC_TASKS -d

Output File Example

TikTok Scraper - Tool to Download Tiktok Video Posts, Collect User trend hashtag music feed metadata, sign URL

You May Also Like

Leave a Reply

Your email address will not be published. Required fields are marked *

one × 4 =