The legal fantasy
The confident, formatted, wrong legal theory that a machine gave someone, and the hour it takes to take it apart.
Here is a new kind of meeting that did not exist three years ago.
Someone comes in with a printout. It is several pages, with headings, bullet points, and a section at the end called "Recommended Next Steps." It explains that they have a strong claim, names the legal doctrine, cites two or three cases, and estimates what the claim is worth. They have read it many times. They have shown it to friends. They are not here to ask whether it is right. They are here to hire someone to do what it says.
It is not right. Sometimes the cases do not exist. More often they exist and say something else. The doctrine is real but does not apply to these facts, or it applies and the remedy is nothing like what the printout promised. The next hour is spent taking the document apart, gently, in front of someone who paid for the hour and wanted to spend it on something else.
Why this is different from the old kind of wrong
People have always come in with wrong ideas about the law. That is normal and it is part of the job. What is new is the confidence, and the source of the confidence. The old wrong idea came from a cousin or a television show and the person held it loosely. The new one came from something that sounds like an expert, never hesitates, and produced the answer in ten seconds, and the person holds it the way you hold something you have been told by an authority.
Judges are seeing the same thing from people who represent themselves. Filings that read like briefs and cite like briefs and are built on cases that do not say what the filing says they say. The judge or a clerk has to check every one. A real part of the court's day now goes to correcting the homework of a machine that is not in the room.
The connection to everything else on this site
I think this is the same disease. A person with a dispute used to have to talk to someone about it. A friend, a lawyer, the other party. Every one of those conversations had some friction in it, and the friction did work. The friend said "are you sure." The lawyer said "that's not how it works." The other party said "that's not what happened." The chatbot says none of those things. It says "you have a strong case," and it says it alone, at night, to a person who has not spoken to anyone.
The fix, again, is not a better machine. It is a person. Before you act on a legal theory a piece of software gave you, say it out loud to a human who will push back. If you do not have one, that is the problem to solve first.