Brave browser logo. Image: Rafael Henrique/SOPA Images/LightRocket via Getty Images, as used by TechCrunch.

Web scraping for AI just landed back in court. On November 2, 2026, The New York Times sued Brave, the company behind the privacy-focused Brave browser. A second suit came the same day from the New York Daily News, the Chicago Tribune and the Orlando Sentinel.

Both suits claim Brave scraped news content and sold access to AI companies. For any agency that builds AI tools on web data, this case is worth a close look. I will keep this simple: what happened, what is claimed, and what we think agencies in India and the USA should do next.

What Happened: The Brave Web Scraping Lawsuits

Here are the verified facts, based on reporting from The Verge and TechCrunch. Both cases were filed in federal court in Manhattan.

Who sued: The New York Times filed one suit. The Daily News, Chicago Tribune and Orlando Sentinel filed a similar one.
Who was sued: Brave Software, founded in 2015. TechCrunch says it has raised roughly $250 million, citing private-market trackers.
Main claim: Brave allegedly scraped copyrighted content and gave AI companies paid access to it.
AI companies named: Anthropic, Perplexity, Mistral, xAI and Cohere, described as customers.
Other claims: A browser feature that bypasses paywalls, and blocking publishers' ads while selling Brave's own.
What the Times wants: An injunction and damages.

One detail stands out. The Times suit says Brave built an index of more than 30 billion web pages using an unidentified crawler. Publishers say that left them unable to block it.

Brave did not immediately respond to requests for comment. So keep in mind that everything in the complaints is an allegation for now. A court has not ruled on any of it.

The Strongest Words in the Complaints

The language is sharp. The Times suit calls Brave a "supplier, dealer, and distributor of stolen copyrighted content" for AI companies. It also says Brave "acts both like a burglar" and an AI "fence."

The second complaint says Brave never asked publishers for permission to access their sites. It also says Brave never paid them. Perplexity's communications chief, Jesse Dwyer, pushed back on the Times. He told The Verge the paper is "overpaying its attorneys and underpaying its journalists."

Diagram of the web scraping data chain alleged in the Brave lawsuits, from publishers to the Brave index to AI companies — The data chain described in the lawsuits. Sources: TechCrunch, The Verge. Diagram: Debugged Pro.

Why This Web Scraping Case Is Different

Publishers have sued AI companies before. The Times has already sued OpenAI and Perplexity over similar claims. What is new here is the target.

Brave is not mainly an AI model company. The suits treat it as a middleman that collected the content and sold access to others. Steven Lieberman, the lawyer for the three other papers, put it bluntly: "Buying stolen goods secondhand doesn't make them legitimate."

That framing matters for agencies. Many of us do not scrape the web ourselves. We plug in a search API or a data provider and build on top. This case asks whether the data supplier, and maybe the buyer, carries the risk.

We do not know how the court will answer that. But the question alone should change how agencies pick and document data sources.

What Web Scraping Means for AI Products Today

Most modern AI products touch web data in one of three ways. Each one carries a different level of risk.

Training data. Content used to train or fine-tune a model. This is where most copyright fights have started.
Grounding or retrieval. Live content pulled in to answer a user question. The Verge says AI companies used the Brave index for training, for answering queries, or both.
Direct crawling. Your own bots collecting pages for a client tool, price tracker or research app.

A small support chatbot that reads a client's own help docs is low risk. A news summary app that pulls paywalled articles through a third-party feed is a very different story.

The Paywall Problem

The paywall claim deserves its own note. The Verge reports that Brave promotes "hundreds of top publishers" as reachable through its "Paywall blocker" feature.

If your product shows full text from paid sources, ask how that text got there. If nobody on the team can answer, that is a problem worth fixing before a client launch.

A New Opt-Out Signal for AI Crawlers

There is a second, related story from a few days earlier. On October 30, The Verge reported on a new shared opt-out signal. OpenAI, Google, Microsoft and Anthropic say they will adopt it.

The signal is a "policy token" that lives in a site's robots.txt file. It was proposed at the IETF by Kevin Kelley, a standards lead at Anthropic. Today, sites have to set preferences for each AI company one by one. This token would cover many at once.

Anthropic: already adopted it.
OpenAI: plans to add it in the coming months.
Google and Microsoft: plan to support it next year.
Meta: signaled support but did not say it will honor it.

Microsoft also says sites will be able to opt out of AI training or AI grounding separately. It adds that sites will still appear in Bing search either way.

Chart showing which AI companies support the shared web scraping opt-out signal in robots.txt — Support for the shared AI opt-out signal, as reported by The Verge on October 30, 2026. Chart: Debugged Pro.

Remember that robots.txt is a voluntary rule. The Robots Exclusion Protocol, RFC 9309, says plainly that these rules are not a form of access authorization. Good crawlers follow them. Bad ones do not.

What Agencies in India and the USA Should Do Now

You do not need to panic or stop using web data. You do need a clear paper trail. Here is the checklist we are using with our own clients.

Map every data source. List each API, dataset, crawler and feed your AI product uses. Note who owns it and what the terms allow.
Read your provider's terms. Check if your search or data vendor promises it has rights to the content. Check what happens if it does not.
Add contract protection. Put data sourcing clauses in client contracts. Ask vendors for indemnity where you can.
Respect robots.txt in your own bots. Follow it, identify your crawler with a clear user agent, and keep crawl logs.
Avoid paywalled content. Unless you hold a licence, do not pull or display full text from paid sources.
Prefer licensed or first-party data. Client documents, licensed feeds and public government data are safer starting points.
Help clients protect their own sites. Review their robots.txt and plan for the new AI opt-out token as support rolls out.

If you want help reviewing an AI build or its data pipeline, our team offers software development consulting for this kind of work.

For Agencies Serving US Clients

Copyright questions around AI training are still being worked out in the USA. The U.S. Copyright Office's AI report is a good, plain starting point. Part 3, on generative AI training, came out as a pre-publication version on May 9, 2025.

US clients in media, publishing and education will ask hard questions about this case. Have your data source map ready before they ask.

For Agencies Serving Indian Clients

Indian agencies often build for US and European clients. That means US court cases can shape your contracts even if your team sits in Pune or Noida.

Treat the strictest client market as your baseline. If a project would worry a US publisher, it should worry you too.

A Quick Example From Our Side

Think about a common request: a client wants an AI assistant that answers questions about industry news. The quick build is to plug in a search API and summarize whatever comes back.

The safer build takes a little longer. We pick sources with clear terms, show short excerpts with links back, and keep logs of what was fetched. The product feels the same to users. The risk for the client is far lower.

The same thinking applies to speed and cost work on AI agents. Our recent post on Claude API message threads covers that side. Trust issues matter just as much, as we saw with deepfake apps slipping past app stores.

The Bigger Picture for Web Scraping and AI

Two things are happening at once. Publishers are suing more players in the AI supply chain. At the same time, big AI firms are agreeing on clearer opt-out signals.

My honest view is this. The era of "grab everything and sort it out later" is ending. Agencies that can show where their data came from will win more trust and more serious clients. The ones that cannot will spend more time with lawyers.

Key Takeaways

The New York Times sued Brave on November 2, 2026 over alleged web scraping for AI.
The Daily News, Chicago Tribune and Orlando Sentinel filed a similar suit the same day.
The suits name Anthropic, Perplexity, Mistral, xAI and Cohere as customers of Brave's index.
The claims are allegations. No court has ruled yet.
OpenAI, Google, Microsoft and Anthropic back a shared AI opt-out signal in robots.txt.
Agencies should map data sources, check vendor terms and add contract protection now.

About the Author

Nitin Kumar is the Lead Project Coordinator at Debugged Pro. He helps teams and clients in India and the USA plan, ship and support AI and software projects. He writes about the practical side of new AI tools.