About
History
The idea of crawling Ajax-based web applications started in 2007 as part of the PhD research of Ali Mesbah working with Arie van Deursen.
The initial idea, written in a technical report titled Exposing the Hidden-Web Induced by Ajax, was to automatically generate a static linked mirror of the dynamic DOM states, to make the web application accessible to search engines.
Later in 2008, the event-based crawling technique was published at ICWE’08 and many applications for Crawljax began to emerge. Extending and using Crawljax for automatically testing modern Web 2.0 applications through structural DOM invariants was the next step, which resulted in an ICSE’09 paper. Furthermore, we have been using Crawljax for security testing (ESEC/FSE’09), regression testing (ICST’10), cross-browser compatibility testing (ICSE’11) and many other interesting domains of web analysis and testing.
Contributors
Ali Mesbah
Ali is in charge of managing the direction of the Crawljax core. His duties consist of providing a project vision and roadmap, planning the scope of release milestones and track progress, designing new features, and writing code and unit tests.
Arie van Deursen
Arie has conceptually contributed to the development and progress of Crawljax, especially for the testing extensions. In addition, he provides code reviews and design patterns and solutions to improve the quality and maintainability of the Core. He also promotes Crawljax by presenting the techniques at various venues and companies.
Danny Roest
Danny has contributed numerous bug fixes, new features and performance enhancements since 1.7. In addition to the work done on the Core, he’s written the API for configuring Crawljax and many plugins such as CrawlOverview, ErrorReport, and RegressionTester.
Stefan Lenselink
Stefan has contributed numerous bug fixes, and performance enhancements. His main focus is on making Crawljax multi-threaded (multi-browser) enabling parallel crawling to boost up the performance and scalability. In addition to the work done on the Core, he’s written a number of plugins such as Benchmark and SFGExporter.
Frank Groeneveld
Frank has contributed numerous bug fixes, and performance enhancements to the Core. His main focus is on detection of dynamic invariants. Frank has also helped to make the Core simpler by moving the integrated proxy to a plugin called Webscarabwrapper.
Cor-Paul Bezemer
Cor-paul worked primarily on spotting security violations in web widgets. He wrote a number of Crawljax 1.7 plugins for detecting DOM violations and HTTP request violations, which are called Diva. He contributed numerous bug fixes, and performance enhancements to the 1.7 Core.