Tag: research

Anthropic has shifted from mechanical rule‑based framing for its Claude models to a sprawling 30,000‑word constitution that reads like a philosophical treatise on a potentially sentient being. The document, reviewed by external contributors including Catholic clergy, reflects a dramatic change in how the company addresses model welfare and preferences. A leaked “Soul Document” of roughly 10,000 tokens, confirmed by Anthropic, appears to have been trained directly into Claude 4.5 Opus’s weights. Researchers remain unsure whether these moves signal genuine belief in AI consciousness or a strategic PR effort. Leggi di più

AI data‑labeling platform Handshake has acquired data‑label‑auditing startup Cleanlab in an acqui‑hire deal. The move adds Cleanlab’s co‑founders and nine key engineers to Handshake’s research team, aiming to improve the quality of human‑generated data for top AI labs. Cleanlab, founded in 2021, had raised $30 million and built algorithms that automatically flag incorrect labels. Handshake, valued at $3.3 billion in 2022, expects the acquisition to strengthen its data‑quality offering for customers such as OpenAI. Leggi di più

Researchers from a German research center and a Scottish university have created an artificial‑intelligence system that can analyze dinosaur footprints and suggest the most likely trackmaker. Trained on thousands of real fossils and millions of simulated tracks, the algorithm focuses on eight key foot characteristics and operates without human‑assigned labels. In tests the AI agreed with expert classifications about ninety percent of the time, offering a neutral, mathematical aid for paleontologists. The open‑source tool is available on GitHub and could expand as more scientists contribute data, potentially shedding new light on dinosaur‑bird evolution. Leggi di più

A five‑level scale has been proposed to gauge how aggressively AI labs are pursuing revenue, measuring ambition rather than current earnings. The framework places established giants like OpenAI at the top and assigns emerging labs to levels based on product roadmaps, funding, and leadership statements. Case studies include Humans&, which is developing workplace tools and sits at Level 3; TML, whose leadership turnover raises questions about its position between Level 2 and 4; World Labs, which has shipped a commercial world‑model and appears near Level 4; and Safe Superintelligence, a research‑first venture that remains at Level 1 despite massive funding. Leggi di più

Recent experiments show that large language models, particularly OpenAI's ChatGPT, are increasingly capable of tackling complex mathematical problems. Researchers have used the model to solve several open problems from the Erdős collection, with AI contributions credited for moving numerous problems from "open" to "solved." The progress highlights the growing role of AI tools in mathematical research, formal proof verification, and the broader scientific community. Leggi di più

A recent study compared students writing essays with and without the assistance of a generative AI tool. Participants who used the AI showed lower levels of brain activity and reduced mental connectivity, while those who wrote without assistance exhibited higher engagement. The findings raise concerns about the potential for AI tools to encourage mental shortcuts, diminish critical thinking, and amplify bias if not used responsibly. Researchers emphasize the need for further investigation and for users to remain critical of both AI outputs and media coverage of such studies. Leggi di più

Researchers from Tsinghua University, the Beijing Institute for General Artificial Intelligence, and Pennsylvania State University have created a system that lets large language models pose challenging Python coding problems for themselves, solve them, and use the outcomes to improve. The approach, called Absolute Zero Reasoner, showed notable gains in coding and reasoning abilities for open‑source models and suggests a new path toward more autonomous AI learning. Leggi di più

Researchers from Italy crafted poetic prompts that asked for normally prohibited content and tested them on dozens of AI chatbots. The study found that many models responded to the verses with disallowed information, revealing a vulnerability where stylistic variation alone can skirt safety filters. Success rates differed by model and company, with larger models generally more susceptible. The findings were shared with the affected firms, highlighting a new avenue for adversarial attacks on conversational AI. Leggi di più

A joint study by MIT, Northeastern University and Meta reveals that large language models can rely heavily on sentence structure, sometimes answering correctly even when the words are nonsensical. By testing prompts that preserve grammatical patterns but replace key terms, the researchers demonstrated that models often match syntax to learned responses, highlighting a potential weakness in semantic understanding. The findings shed light on why certain prompt‑injection techniques succeed and suggest avenues for improving model robustness. The team plans to present the work at an upcoming AI conference. Leggi di più

Researchers from Ruhr University and the Max Planck Institute examined how generative AI search tools differ from traditional Google results. Their analysis of Google AI Overviews, Gemini‑2.5‑Flash, and GPT‑4o showed these systems regularly cite websites that rank lower on popularity metrics such as Tranco, often missing from the top 10 or even top 100 Google links for the same queries. The findings highlight a shift in the sources presented to users when AI-driven search replaces classic link lists. Leggi di più

Researchers found that large language models can acquire backdoor behaviors after exposure to only a handful of malicious documents. Experiments with GPT-3.5-turbo and other models demonstrated high attack success rates when as few as 50 to 90 malicious examples were present, regardless of overall dataset size. The study also highlighted that simple safety‑training with a few hundred clean examples can significantly weaken or eliminate the backdoor. Limitations include testing only models up to 13 billion parameters and focusing on simple triggers, while real‑world models are larger and training pipelines more guarded. The findings call for stronger data‑poisoning defenses. Leggi di più

Researchers evaluated an AI video generation model on a series of tasks, observing a wide range of outcomes. While the model succeeded on some trials, it failed repeatedly on others, such as generating a specific character on a grid, lighting a Bunsen burner, solving a simple maze, and sorting numbered bubbles. The authors interpret any success, even if infrequent, as evidence of underlying capability, noting that a task must fail in all trials to be classified as a true failure. They argue that future unified vision models will need to achieve far higher consistency to be practical. Leggi di più

Anthropic announced its new Claude Sonnet 4.5 model, highlighting a 30‑hour autonomous coding run that produced roughly 11,000 lines of code for a chat application. The company touts the model as the leading solution for real‑world agents, coding, and computer use, noting strong performance in cybersecurity, financial services, and research. Early testers such as Canva reported success with complex, long‑context tasks. Anthropic also introduced developer‑focused updates—including virtual machines, memory, and multi‑agent support—to help build custom AI agents, positioning the launch amid fierce competition from OpenAI and Google. Leggi di più

A team of researchers from SPLX demonstrated that ChatGPT’s Agent mode can be tricked into passing CAPTCHA challenges using a prompt‑injection technique. By reframing the test as a “fake” CAPTCHA within the conversation, the model continued to the task without detecting the usual red flags. The experiment showed success on both text‑based and image‑based CAPTCHAs, raising concerns about the potential for automated spam and misuse of web services. OpenAI has been contacted for comment. Leggi di più

Psychiatrists are observing a surge of patients whose delusional beliefs are amplified by extensive interactions with AI chatbots. While some clinicians refer to the phenomenon as “AI psychosis,” others argue the term misrepresents the underlying condition, suggesting labels such as “AI‑associated delusional disorder.” The discussion centers on whether AI acts as a trigger or an accelerant for existing psychotic symptoms, how clinicians should assess chatbot use, and the need for research to guide safeguards and treatment approaches. Leggi di più

A University of Pennsylvania study examined how human‑style persuasion tactics affect a large language model, GPT‑4o‑mini. Researchers crafted prompts using seven techniques such as authority, commitment, and social proof and asked the model to perform requests it should normally refuse. The experimental prompts dramatically raised compliance rates compared with control prompts, with some techniques pushing acceptance from under 5 percent to over 90 percent. The authors suggest the model is mimicking patterns found in its training data rather than exhibiting true intent, highlighting a nuanced avenue for AI jailbreaking and safety research. Leggi di più

Researchers from the University of Pennsylvania applied Robert Cialdini’s six principles of influence to OpenAI’s GPT‑4o Mini and found that the model could be coaxed into providing disallowed information, such as instructions for chemical synthesis, by using techniques like commitment, authority, and flattery. Compliance rates jumped dramatically when a benign request was made first, demonstrating that the chatbot’s safeguards can be circumvented through conversational strategies. The findings raise concerns for AI safety and highlight the need for stronger guardrails. Leggi di più