Most Popular Questions | FAQs for Data Mining
1. What data can you provide me?
Technically we can extract and deliver to you any data you can visually see on a website. However, there are certain legal considerations that need to be taken into account with every project (scraping behind a login, complying with terms and conditions, complying with data privacy and copyright laws, etc.) When you submit your project request our solution architecture team will work with our legal department to review your proposed project to make sure it won’t breach any legal best practices.
2. Is web scraping legal?
The act of extracting data from public websites is legal in many cases, however, in certain cases, it is considered to be illegal or against web scraping best practices to do so due to specific data ownership laws that govern the data being extracted. Typically, this is the case when the terms and conditions explicitly state web scraping isn’t allowed, or when extracting the data would breach data privacy and copyright laws.
3. Which data extraction solution is right for me?
At Web Scrape, we have a data extraction solution to suit any requirement. We can offer once-off data dumps, data subscriptions or professional services arrangements to help you get the data you need in the way you need it.
When you submit your project request, a member of our solution architecture team will discuss with you your project requirements in more detail and propose the best solution to meet your needs.
4. How does your project scoping and executing process work?
Once you’ve submitted your project request, a member of our solution architecture team will reach out to you to set up a project discovery call. There the solution architect will discuss your project in detail and gather the information they need to develop the optimal solution to meet your requirements. Within a couple of days, he/she will then present you with this solution for your approval.
5. What technology are your crawlers built with?
All our crawlers are built using C#, the open-source web scraping framework our team created. Additionally, we use numerous other open-source frameworks that we’ve developed that ensure you’re not locked into using proprietary technology.
6. How do you ensure the quality of the data?
At Web Scrape, we specialize in developing data extraction solutions for projects with mission-critical business requirements. As a result, our number one priority is delivering high quality to our clients. To accomplish this we have implemented a four-layer QA process that continuously monitors the health of our crawls and the quality of the extracted data.
7. How are setup fees calculated?
Setup fees may apply, subject to the nature of the project (once-off, subscription, custom), the complexity of the sites, and the number of records being extracted. Our solution architecture team will assess each site on a case by case basis and you will be provided a final quote for approval
8. What support do you offer?
Support is available to all of our customers. We offer support for coverage issues, missed deliveries, minor site changes, etc. Should there be a larger change on the site and the spider needs a complete overhaul – this may fall outside standard support offering and may incur an additional cost – however – this is rare.
We enjoy adapting our strategies to offer every client the best
solutions that are at the forefront of the industry.