• DocumentCode
    2761816
  • Title

    PyBot: An Algorithm for Web Crawling

  • Author

    Leng, Alex Goh Kwang ; Ravi, Kumar P. ; Singh, Ashutosh Kumar ; Dash, Rajendra Kumar

  • Author_Institution
    Dept. of Electr. & Comput. Eng., Curtin Univ., Miri, Malaysia
  • fYear
    2011
  • fDate
    8-10 Dec. 2011
  • Firstpage
    1
  • Lastpage
    6
  • Abstract
    PyBot is a Web Crawler developed in Python to crawl the Web using Breadth First Search (BFS). The success of the World Wide Web (WWW), which itself built on the open internet, has changed the way how human share and exchange information and ideas. With the explosive growth of the dynamic Web, users have to spend much time just to retrieve a small portion of the information from the Web. The birth of the search engines have made human lives easier by simply taking them to the resources they want. Web crawler is a program used by search engines to retrieve information from the World Wide Web in an automated manner.
  • Keywords
    Internet; information retrieval; search engines; PyBot; Python; Web crawler; World Wide Web; breadth first search; information retrieval; open Internet; search engines; Crawlers; Engines; Indexes; Robots; Search engines; Web pages;
  • fLanguage
    English
  • Publisher
    ieee
  • Conference_Titel
    Nanoscience, Technology and Societal Implications (NSTSI), 2011 International Conference on
  • Conference_Location
    Bhubaneswar
  • Print_ISBN
    978-1-4577-2035-2
  • Type

    conf

  • DOI
    10.1109/NSTSI.2011.6111993
  • Filename
    6111993
    • Link To Document
    https://search.isc.ac/dl/search/defaultta.aspx?DTC=49&DC=2761816