wiki.ziemers.de

ziemer's informatik Wiki

Benutzer-Werkzeuge

Webseiten-Werkzeuge

Zuletzt angesehen:
wiki:software:beuthbot:webscraper

Unterschiede

Hier werden die Unterschiede zwischen zwei Versionen angezeigt.

Link zu dieser Vergleichsansicht

Beide Seiten der vorigen Revision Vorhergehende Überarbeitung
Nächste Überarbeitung 		Vorhergehende Überarbeitung
wiki:software:beuthbot:webscraper [12.12.2019 13:19]
Timo Bruns [Getting Started] 		wiki:software:beuthbot:webscraper [24.01.2020 14:26] (aktuell)
Timo Bruns [Authors]
Zeile 1: Zeile 1:
-===== Scraper Doku =====+===== Scraper Documentation =====
    
  
Zeile 5: Zeile 5:
  
 ==== Table of content ==== ==== Table of content ====
- + - [[wiki:software:beuthbot:webscraper|Scraper Doku]] 
-  - [[wiki:software:beuthbot:webscraper|Scraper Doku]] + - [[wiki:software:beuthbot:webscraper#table_of_content|Table Of Content]] 
-  - [[wiki:software:beuthbot:webscraper#table_of_content|Table of content]] + - [[wiki:software:beuthbot:webscraper#getting_started|Getting Started]] 
-  - [[wiki:software:beuthbot:webscraper#getting_started|Getting Started]] + -[[wiki:software:beuthbot:webscraper#prerequisites|Prerequisites]] 
-  - [[wiki:software:beuthbot:webscraper#overview|Overview]] + -[[wiki:software:beuthbot:webscraper#installing |Installing ]] 
-  - [[wiki:software:beuthbot:webscraper#structure|Structure]] + - [[wiki:software:beuthbot:webscraper#overview|Overview]] 
-  - [[wiki:software:beuthbot:webscraper#functionalities|Functionalities]] + - [[wiki:software:beuthbot:webscraper#structure|Structure]] 
-   -[[wiki:software:beuthbot:webscraper#study_rooms|Study Rooms]] + - [[wiki:software:beuthbot:webscraper#functionalities|Functionalities]] 
-  - [[wiki:software:beuthbot:webscraper#further_development|Further Development]] + -[[wiki:software:beuthbot:webscraper#study_rooms|Study Rooms]] 
-  - [[wiki:software:beuthbot:webscraper#built_with|Built With]] + - [[wiki:software:beuthbot:webscraper#further_development|Further Development]] 
-  - [[wiki:software:beuthbot:webscraper#versioning|Versioning]] + - [[wiki:software:beuthbot:webscraper#built_with|Built With]] 
-  - [[wiki:software:beuthbot:webscraper#authors|Authors]]+ - [[wiki:software:beuthbot:webscraper#versioning|Versioning]] 
 + - [[wiki:software:beuthbot:webscraper#authors|Authors]]
  
 ==== Getting Started ==== ==== Getting Started ====
Zeile 25: Zeile 26:
  
 You will need a current version of [[https://nodejs.org/en/|node & npm]]. You will need a current version of [[https://nodejs.org/en/|node & npm]].
 +
 +
  
 === Installing === === Installing ===
Zeile 42: Zeile 45:
 ==== Overview ==== ==== Overview ====
  
-The bot is basically a _Node_-_Express_-Backend. Incoming requests are checked and specifically handled.+The bot is basically a //Node//-//Express//-Backend. Incoming requests are checked and specifically handled.
  
 ==== Structure ==== ==== Structure ====
  
-The bot is separated into two files. `index.js` contains the fundamental logic. At the moment there is only one ressource but we expect to expand this with more ressources. A ressource is represented by a _route_. If the user request the lists of study rooms at our university, the script notices the request at the specified route and prepares a _JSON_ response.+The bot is separated into two files. ''index.js'' contains the fundamental logic. At the moment there is only one ressource but we expect to expand this with more ressources. A ressource is represented by a //route//. If the user request the lists of study rooms at our university, the script notices the request at the specified route and prepares a //JSON// response.
  
-The second script, `scrape.js`, takes care of the actual web scraping. The given URL is requested with _axios_ and then parsed with _cheerio_.+The second script, ''scrape.js'', takes care of the actual web scraping. The given URL is requested with //axios// and then parsed with //cheerio//.
  
 ==== Functionalities ==== ==== Functionalities ====
Zeile 54: Zeile 57:
 === Study Rooms === === Study Rooms ===
  
-When the resource is requested we scrape the [ASTA Website]('https://asta.studis-bht.de/service/lernraeume/') and try to return a list of available rooms, that are provided for students, from our university.+When the resource is requested we scrape the[[https://asta.studis-bht.de/service/lernraeume/|ASTA Website]] and try to return a list of available rooms, that are provided for students, from our university.
  
 ==== Further Development ==== ==== Further Development ====
  
-Add a new route for every ressource in `index.js` and prepare functions in `scrape.js` to scrape the requested data from given Websites.+Add a new route for every ressource in ''index.js'' and prepare functions in ''scrape.js'' to scrape the requested data from given Websites.
  
 ====  Built With ==== ====  Built With ====
  
-- [Node.js](https://nodejs.org/en/) \\  +- [[https://nodejs.org/en/|Node.js]]\\ 
-- [Express.js](https://expressjs.com/) \\  +- [[https://expressjs.com/|Express.js]]\\ 
-- [Axios](https://github.com/axios/axios) \\  +- [[https://github.com/axios/axios|Axios]]\\ 
-- [Cheerio](https://github.com/cheeriojs/cheerio)\\ +- [[https://github.com/cheeriojs/cheerio|Cheerio]]\\ 
  
 ==== Versioning ==== ==== Versioning ====
  
-We use [SemVer](http://semver.org/) for versioning. For the versions available, see the [tags on this repository](https://github.com/T0biWan/scrape-js/tags).+We use [[http://semver.org/|SemVer]] for versioning. For the versions available, see the [[https://github.com/beuthbot/scraper/tags|tags on this repository]].
  
 ==== Authors ==== ==== Authors ====
  
-- **Tobias Klatt** - _Initial work_ - [GitHub](https://github.com/T0biWan/) +- **Tobias Klatt** - //Initial work// - [[https://github.com/T0biWan/|GitHub]]
- +
-See also the list of [contributors](https://github.com/T0biWan/scrape-js/graphs/contributors) who participated in this project.+
  
 +See also the list of [[https://github.com/beuthbot/scraper/graphs/contributors|contributors]] who participated in this project.
 +<WRAP pagebreak></WRAP>
wiki/software/beuthbot/webscraper.1576153173.txt.gz · Zuletzt geändert: 12.12.2019 13:19 von Timo Bruns

Seiten-Werkzeuge