RDocumentation
Moon
Learn R
Search all packages and functions
Rcrawler (version 0.1.9-1)
Web Crawler and Scraper
Description
Performs parallel web crawling and web scraping. It is designed to crawl, parse and store web pages to produce data that can be directly used for analysis application. For details see Khalil and Fakir (2017)
.
Copy Link
Copy
Link to current version
Version
Version
0.1.9-1
0.1.8-0
0.1.7-0
0.1.5
0.1.4
0.1.3
0.1.2
0.1.1
0.1
Down Chevron
Install
install.packages('Rcrawler')
Monthly Downloads
620
Version
0.1.9-1
License
GPL (>= 2)
Issues
33
Pull Requests
6
Stars
347
Forks
95
Repository
https://github.com/salimk/Rcrawler/
Maintainer
Salim Khalil
Last Published
November 11th, 2018
Functions in Rcrawler (0.1.9-1)
Search functions
Getencoding
Getencoding
LinkExtractor
LinkExtractor
LoginSession
Open a logged in Session
install_browser
Install PhantomJS webdriver
RobotParser
RobotParser fetch and parse robots.txt
run_browser
Start up web driver process on localhost, with a random port
Rcrawler
Rcrawler
stop_browser
Stop web driver process and Remove its Object
browser_path
Return browser (webdriver) location path
ContentScraper
ContentScraper
LinkNormalization
Link Normalization
Linkparameters
Get the list of parameters and values from an URL
Linkparamsfilter
Link parameters filter
LoadHTMLFiles
LoadHTMLFiles @rdname LoadHTMLFiles
ListProjects
ListProjects
Drv_fetchpage
Fetch page using web driver/Session