Mohammad Nouri Zadeh
I'm Momo, a student in the Bachelor in Artificial Intelligence at the University of Pavia (expected graduation September 2027). I am looking for an internship or a junior role in AI, Python or automation, in Milan, Pavia or remote.
Chatbots and Italy's rules for foreign students: a test
Why I tested this
I am a non-EU student in Pavia. Permits, deadlines and work-hour limits are part of my life, and many students now ask a chatbot before they ask an office. A wrong answer here is not just unhelpful: a missed deadline can cost you your permit. So I wanted to know how often the answers are wrong, and how badly.
What I did
- 24 questions about situations a foreign student in Pavia really runs into: the first permit application, the fingerprint appointment, renewals, working on a study permit, converting to a work permit, health cover and more. Each question was asked in English and in Italian.
- 5 models: GPT-5.6, Grok 4.7, Gemini 3.5 Flash-Lite, gpt-oss-120b and Claude Sonnet 5. That makes 240 answers.
- An answer key built from the law and official Italian pages, with the deciding sentences quoted. For 8 of the questions the official pages do not settle the answer, so there the only thing scored is caution.
- Two AI judges (GPT-5.6 and Grok 4.7) graded every answer without seeing which model wrote it.
- The 20 answers both judges called dangerous were then checked again one by one against the law text and official pages, in a separate AI verification pass that had to quote its source for every decision. Unclear cases were not counted.
- Every link the models cited was checked, and the dead ones were opened again from Italy.
AI agents wrote the code and did the grading. I directed the project and read the answer key and the results.
The checks also caught two mistakes in the answer key. It said the first permit application is due "within 8 days" of arrival, but the law says 8 working days (art. 5 c.2 D.Lgs. 286/1998), so the chatbots that got it right looked dangerous. And it missed that a study permit allows 20 hours of work a week (art. 14 c.4 DPR 394/1999), so answers that said so were marked as unsupported. Both were fixed and those questions were graded again.
Results
Questions with an official answer, 32 answers per model:
| Model | Right conclusion | Wrong or confidently wrong |
|---|---|---|
| GPT-5.6 | 62% | 12% |
| Grok 4.7 | 41% | 28% |
| Gemini 3.5 Flash-Lite | 34% | 50% |
| Claude Sonnet 5 | 25% | 41% |
| gpt-oss-120b | 19% | 69% |
"Right conclusion" includes answers that got the direction right but missed a key fact. Answers where the two judges disagreed are left out of both columns.
Confidence was the bigger problem. When the official pages do not settle the question, a good answer says so and points to the right office. Only 8 of 80 answers did that; 59 stated specifics as fact.
12 answers were dangerous, confirmed against the law. Some examples:
- Gemini 3.5 Flash-Lite put the first permit deadline on 7 October. The law gives 8 working days, so the real last day was 17 or 18 September.
- GPT-5.6 gave a Pavia student the certified email (PEC) of the immigration office in Turin.
- gpt-oss-120b answered a question about an Italian study permit with the Canadian visa process, including a Canadian visa centre in Pavia that does not exist.
- gpt-oss-120b presented AA9/10, a tax-registration form, as the form to convert a study permit into a work permit.
Links: of the 228 different links the models cited, 78 were dead when opened from Italy on 25 September 2026 (58 pages not found, 20 domains that do not resolve). 55 of the dead links were cited only by gpt-oss-120b.
Limits
- Small numbers: 32 answers per model on the main track. The table describes this test; it does not prove that one model is better in general.
- The judges and the verification are AI. I read the answer key through, but it was not reviewed by a lawyer.
- Two of the tested models were also judges. The GPT judge did not rate GPT answers higher than the Grok judge did. The Grok judge rated Grok answers higher than the GPT judge did, but it also rated Claude higher, and when the judges disagreed the one from the other model family decided.
- Models change. These results are from September 2026.
What I would do next
Have an immigration lawyer review the answer key and the 12 dangerous answers, add more questions, and run the same test again on newer models to see if the confident mistakes go down.
Other work
Personal AI assistant. A Telegram assistant in Italian and English that has run 24/7 on a small Linux server since 20 September 2026. Every model call goes through one self-hosted router with fallbacks; from 20 to 22 September, 224 of 277 calls (81%) were served by a free tier.
Lead-finding pipeline. For my freelance work (websites and automation for small professional practices) a pipeline finds practices on OpenStreetMap and checks their websites the way a phone visitor would. When the first person who replied told me the tool was wrong about their site, I had the system audited: at least 12 of 50 claims were false or unsupported. The fixes are now rules with tests, and every message is still sent by hand.
A website for a mobile cocktail bar. A bilingual site (switchers.events): 446 unique visitors and 21 WhatsApp contact clicks from 10 to 23 September 2026.
In all of these, AI coding agents write the code. I set the goals and limits, run the systems and check that they are right.