In this edition of the Data Dispatch, we'll take a look at Artificial Intelligence (AI) and how it relates to Dataprovider.com. Below, you can read about how we track AI and its usage with structured web data, as well as how we use it to improve and upgrade our services. Check out our latest blog insights, and find April's three interesting links at the end—this month, it's all about optical pleasures.
Dataprovider.com and AI
Tracking AI with structured web data
As an increasing part of daily workflows, the presence of AI and generative tools has grown on the contemporary web. And, of course, the structured web data in our database has reflected these changes.
How do we track AI with structured web data? We look where the changes are most visible. In the graph below you can see the kind of information that traffic data reveals—ChatGPT is the most popular LLM, DeepSeek's peak was in January but has declined and stabilized since, and the fact that traffic to genAI sites consistently dips on the weekend shows that for most, it's a workday tool.
Besides that, we've explored the data behind robots.txt files and llms.txt files across the web, dissecting what mentions of the web crawlers of AI companies might mean in our blog insights. And the boom of .ai domains has shown that the business and cultural consciousness of AI trends has truly marked the digital landscape.
Dataprovider.com and AI
Using AI to structure web data
Beyond tracking the rise of AI online, our teams use it to improve and update our products. For example, the Summaries feature adds an AI-generated description to entries in our database, summarizing a website's purpose, its business industry, products and services, and other relevant information at a glance.
More generally, many of the data fields in the Search Engine are underpinned by machine learning: our data annotators build a foundation of classifications (e.g., regarding business models, industry codes, or e-commerce status) and our algorithms use artificial intelligence to classify web data on the macro level. There's plenty of room to incorporate responsible AI usage into our processes, and we look forward to future developments.
In our monthly crawl of every domain on the web, we come across all kinds of interesting things. Websites are a creative outlet for many kinds of artists, and experimental sites can be visually enchanting galleries. We've collected a few of our favorites, so we hope you enjoy this month's digital exploration of optical delights.
RIVER by Max Bittker — a never-ending flow of images, a river of visual wonders. Dive in and surf until you find an image you like, then click to see the river expand with reminiscent art, then do it again. Max Bittker calls this an 'exploration engine', built to challenge the one-dimensional scroll, engagement-driven predictions, and algorithmic experiences too common on the internet today.
DVEIN — from a director duo, DVEIN is a portfolio like few others on the internet: a co-creative showcase of their work, but also the contributions of visitors to their site. Enter the artwork and explore remarkable images, or take the time to draw on their walls (as you enjoy what others have created too).
Infinite View by PDIA — each year, thousands of images—photographs, artworks, posters, diagrams—enter the public domain. Whether a 100-year copyright expires or copyright is simply released, these visual treats become free to view, use, and reuse. But where to find them? Thanks to the Public Domain Review, you can access and explore the images in an incredibly satisfying infinite view.
