PromptLayer

PromptLayer AI

PromptLayer AI

PromptLayer AI

PromptLayer AI

PromptLayer AI

PromptLayer AI

PromptLayer AI

PromptLayer AI

PromptLayer AI

PromptLayer AI

I built an open-source Desktop App that gives your AI persistent memory across all platforms (100% Local SQLite, Zero-Docker)

Hey everyone, A few weeks ago I shared the CLI version of my project, ArcRift, on Reddit. After listening to your feedback—specifically the requests to remove heavy Docker dependencies and make it easier to install—I have just released the v1.6.1 Desktop App. If you regularly use LLMs for coding or research, you know the frustration of "amnesia." Every time you open a new chat, you have to painstakingly copy and paste your project structure and previous context just to get the AI up to speed. ArcRift is a 100% offline, local-first RAG and memory layer. It bridges the gap between your AI web chats (like Claude and ChatGPT) and your local tools (like Cursor or Claude Code) using a unified local database. I wanted something lightweight that did not require pulling Docker containers or subscribing to third-party memory APIs. It now runs as a native Tauri desktop app in your system tray, powered completely by local Ollama instances and a local SQLite database. We just launched a live website that outlines the details and demonstrates the features in action: Website: https://arcrift.vercel.app/ Codebase: https://github.com/Eshaan-Nair/ArcRift How it works & Core Features: Seamless Integration: The Chrome extension silently intercepts your prompts, surgically retrieves exactly the sentences relevant to your question from your database, and injects them before the prompt is sent to the LLM. Hybrid Search Retrieval: Uses sqlite-vec (with nomic-embed-text locally) + FTS5 keyword prefix matching to instantly find your past context. Knowledge Graph Extraction: An offline task queue uses a local LLM to extract entity relationships from your chats, mapping out a graph of your projects over time. Direct Codebase Indexing: The new Desktop App allows ArcRift to scan and index your actual project files into the graph, bridging the gap between your chat memory and your actual code architecture. Total Privacy (PII Redaction): The extension aggressively scrubs JWTs, API keys, emails, and IPs before data is even saved to your local disk. The extension works natively with Claude.ai, ChatGPT, DeepSeek, Gemini, Grok, and Mistral. If you save a conversation in ChatGPT today, you can instantly recall that exact context in Claude tomorrow. ArcRift is completely open-source (MIT). You can download the new .exe installer directly from the GitHub releases page. If you find this useful for your daily workflow, PRs are very welcome, and a star on GitHub helps the project get discovered! submitted by /u/Better-Platypus-3420 [link] [comments]

What actually is "Prompt Engineering"?

I've been thinking about this lately because I feel like people use the term "prompt engineering" to describe two very different things. On one end, you have what most people are familiar with: A person opens ChatGPT, Claude, Gemini, etc., and writes a carefully structured prompt. They define a role, provide context, establish goals, set constraints, maybe include examples, and iterate until they get the output they want. Most people seem to call this prompt engineering. But on the other end, when I'm building AI systems, prompt engineering looks completely different. The prompt isn't really a prompt anymore. It's much more of a dynamic pipeline. Variables are injected from databases, user input, APIs, previous conversations, tools, memory systems, retrieval systems, business rules, and workflow state. Decision trees determine which instructions are included and which are excluded. Prompts become assembled in real time based on context. In some cases, the "prompt" is really just an orchestration layer made up of dozens of smaller prompts, conditionals, guardrails, routing decisions, and context windows. At that point, are we still talking about prompt engineering? Or are we actually talking about system design, context engineering, workflow engineering, orchestration, or something else entirely? Personally, I see prompt engineering as a spectrum: Level 1: Writing a better prompt. Level 2: Designing reusable prompt templates. Level 3: Building dynamic prompts with variables and context injection. Level 4: Engineering entire prompt-driven systems with routing, memory, tools, retrieval, and decision logic. Curious where others draw the line. When you hear "prompt engineering," are you thinking about writing prompts, building workflows, designing agent systems, or all of the above? Has the term become too broad to be useful? submitted by /u/Early-Matter-8123 [link] [comments]

As a traumatized 4.7 user, Opus 4.8 is a breath of fresh air. 4.8 just one-shot the conversion of an Android app into the iOS counterpart.

I vibecode apps, I love it, I put a lot of efforts and care into my apps, I develop them primarily to solve real problems I personally face, but I share them on the stores because why not. I decided I wanted to build a Reddit counterpart, the app is already up and running on the Play Store, and from experience, I made sure that it is easily convertible to iOS, meaning that 95% can be literally copy-pasted, so to be honest, the job is straightforward. The layers of complexity comes with Apple's own specific stuff, like the provisions that you need to make (create an app bundle, get the certificate for it, keys for apple signup and notifications, secrets to deploy to backend, google auth), and to me, it can get a bit confusing. With 4.6, it was a matter of trial and error, we do what we can, we build in the test envrionment and find out what happens lol. It was nice, by the end of the process, it ends up finding all gaps and properly guiding me on how to fill them and correct them. So, I asked 4.8 to do the same, I just copy-pasted the app, created a new repo and sent it a lengthy prompt basically telling it that we want to convert this app into an iOS one, pay attention to what I described above, and oh my lord. Things that 4.6 consistently messed up, Google signups, app.json variables...etc were all caught, I was surprised, like there is one annoying thing in app.json which something like "ipad.compatible = ?" and 4.6, for some reason, always set that to true, so when I submit the iOS app, Apple's like, "EXCUSE ME ☝🤓 where are the iPad screenshots?" but 4.8 caught it and was like, "hey, do we want this app to be in iPad?" and then it listed me a step-by-step dumbed-down plan on what to do and how to properly prepare the provisions for the app to be production-ready. Then it caught the icons not being compatible with iOS, it did a pass and corrected those, then it caught the "return" button on iOS devices (Android doesn't need it because it can gesture or native navigation bar on OS-level) and fixed that, and then it asked me to build and test, and it was insane, it basically worked from the first try, and I'm baffled. Highly recommend it. I have it set to extra high in efforts, so nothing crazy, and consumption has been steady. submitted by /u/Sweet_Brief6914 [link] [comments]

New to coding, what’s the workflow you recommend? This is mine…

I’m a non-developer founder building a SaaS product (web app, TypeScript/Next.js/Postgres stack) mostly through Claude. I have decent architectural intuition but I don’t write code by hand, so I lean heavily on Claude for implementation and on a docs-first process to keep things solid. The workflow I’ve ended up with, over a few months: - Claude Code does the actual implementation, one step at a time. - I run a second Claude chat as an “orchestrator” that drafts the prompts/plans and reviews the code before it ships. - I run a third Claude chat as a “cross-check reviewer” that independently verifies the diff against the plan before I commit. - I’m the one who actually runs every git push, after both review layers sign off. On top of that I keep architecture decision records (ADRs), a running project-state doc, and a “patterns” file where I write down recurring lessons (e.g. how to avoid a class of editing bug, when to bundle vs split commits). It catches a lot of real issues before they ship. But it’s also slow, some days feel heavier on review ceremony and documentation than on actual code progress. Questions for people who’ve built more than me: 1. Is multi-agent review (one model implements, others review) worth it, or is it overkill for a solo project? 2. How much process is right for a non-developer who wants solid code but also needs to actually ship? 3. What does your Claude-assisted workflow look like, and what would you cut from mine? Genuinely open to “you’re overthinking this.” Trying to find the right balance. Thanks. submitted by /u/sorinmx [link] [comments]

the hard part of an automated sprint review isn't the summary, it's the join

Spent a while trying to get one sprint digest out of linear, github, and slack and the summarization was never the hard part. the join is. linear calls it ENG-1432, github calls it PR #890, the incident is a slack thread with no shared id at all. a chat-window model summarizes each source fine but it can't reconcile that the PR closed the issue that caused the incident, because it never holds all three at once with the relationships intact. what actually moved this for me was a desktop agent (Runner) where the connectors aren't thin rest wrappers. they do association traversal, so the github side already knows which PR references which linear issue, and the digest comes out as 'this deploy shipped these issues, one reopened after an incident' instead of three disconnected bullet lists. deploy status and incident notes in the same view is where it gets useful and also where most tool-calling setups quietly fall apart, the model guesses the cross-references instead of resolving them. if you wired this up with raw function calling, did the entity resolution end up living in the prompt or down in the tool layer? written with ai submitted by /u/Deep_Ad1959 [link] [comments]

🚀 Prompt Logic Gates (PLG): Are Prompts Becoming Systems?

GitHub: Prompt-Logic-Gates-PLG Over the past few days, I've shared my research project Prompt Logic Gates (PLG) and received a lot of interesting feedback. Some people loved the idea, some were skeptical, and many raised valid questions. The most common reaction was: > "Natural language is already the abstraction layer. Why add logic gates?" That's a fair question. My goal isn't to replace natural language prompting. In fact, natural language remains at the center of PLG. The idea is to explore what happens when prompts stop being a single request and start becoming systems. The Problem When we write prompts, we're converting our ideas, requirements, constraints, and expectations into text. For simple tasks, this works perfectly. But as prompts grow, they often include: Multiple objectives Business rules Style constraints Context dependencies Exclusions Fallback instructions Tool orchestration At that point, prompts become harder to maintain. Contradictions appear. Priorities become unclear. Context gets mixed together. The prompt is still text, but the complexity starts to resemble a system. What is PLG? Prompt Logic Gates (PLG) is a visual prompt engineering experiment that explores whether prompts can be organized before being sent to an AI model. Instead of writing one giant prompt, users create prompt components and connect them using semantic logic gates. The AI then analyzes the graph and compiles a final structured prompt. How It Works AND Gate When multiple instructions exist, the system evaluates them against the current context and determines which instruction is more foundational. The higher-priority instruction is applied first. OR Gate When multiple options are available, the system selects the most contextually relevant option instead of blindly including everything. NOT Gate Defines exclusions and negative constraints. It explicitly tells the system what should not be done, reducing contradictions and ambiguity. Ask Questions Gate If the system detects missing information or uncertainty, it asks follow-up questions before generating the final prompt. Addressing Common Criticisms "This is just block coding." Not exactly. The goal isn't to create a programming language for prompts. The nodes still contain natural language. The visual layer only helps express relationships between prompt components. "Prompts aren't code." I agree. But once prompts include branching decisions, reusable components, exclusions, fallback behavior, memory, and tool orchestration, they start behaving less like a sentence and more like a system. PLG is exploring whether that hidden structure can be represented more explicitly. "Visual prompt engineering may be harder to debug." That's a valid concern. Visual doesn't automatically mean better. One of the main goals of this project is to test whether visual organization actually improves maintainability, reusability, and prompt consistency—or whether it simply makes the same complexity look different. "The future is promptless AI." Maybe. But today's AI systems still rely heavily on instructions, context, constraints, and reasoning frameworks. Even if prompts eventually disappear, the underlying problem of organizing intent, requirements, and context may still exist. Why I'm Building This This project started because I was facing problems in my own prompting workflow. I wanted a way to organize ideas, constraints, and instructions more systematically instead of continuously rewriting large prompts. PLG isn't trying to solve every problem in AI. It's a research experiment exploring one question: > At what point does a prompt stop being "just text" and start behaving like a system that benefits from structure, organization, and validation? I don't know the answer yet. That's exactly why I'm building the prototype and testing it. If the idea turns out to be useful, great. If it doesn't, I'll still learn something valuable about how humans interact with AI systems. I'd love to hear more thoughts, criticism, and feedback from the community. submitted by /u/withsj [link] [comments]

Claude Code Source Deep Dive (Part 6) — Tool-Call Loop Self-Repair Core && End-to-End Query Pipeline Flow

Reader’s Note On March 31, 2026, the Claude Code package Anthropic published to npm accidentally included .map files that can be reverse-engineered to recover source code. Because the source maps pointed to the original TypeScript sources, these 512,000 lines of TypeScript finally put everything on the table: how a top-tier AI coding agent organizes context, calls tools, manages multiple agents, and even hides easter eggs. I read the source from the entrypoint all the way through prompts, the task system, the tool layer, and hidden features. I will continue to deconstruct the codebase and provide in-depth analysis of the engineering architecture behind Claude Code. Part IV: Tool-Call Loop Self-Repair Core Mechanism 4.1 Core Principle Claude Code's "auto bug-fixing" capability is fundamentally a tool-call feedback loop: Claude generates tool_use ↓ Tool executes (success or failure) ↓ tool_result returned to Claude (with is_error flag) ↓ Claude sees the error message in the next round ↓ Analyze cause → try new strategy ↓ Call tool again → loop continues Key design: errors and successes use exactly the same message format. The only difference is is_error: true: // Successful tool_result { type: 'tool_result', tool_use_id: 'call_abc', content: 'file content...', is_error: false } // Failed tool_result { type: 'tool_result', tool_use_id: 'call_abc', content: 'Error: File not found', is_error: true } 4.2 Key Guidance in the System Prompt If an approach fails, diagnose why before switching tactics—read the error, check your assumptions, try a focused fix. Don't retry the identical action blindly, but don't abandon a viable approach after a single failure either. 4.3 Four-Layer Error Recovery Strategy Layer 1: Prompt-Too-Long recovery PTL error → Strategy 1: context-collapse drain → Strategy 2: reactive compact (summarize history) → Strategy 3: report error to user Layer 2: Output token limit recovery Limit hit → Strategy 1: escalate from 8K to 64K (ESCALATED_MAX_TOKENS) → Strategy 2: recovery message "Output token limit hit. Resume directly..." → Strategy 3: give up after at most 3 times Layer 3: Model overload fallback Consecutive 529 errors (3x) → switch to fallbackModel → discard failed attempt result → retry with backup model Layer 4: Natural recovery from tool errors Tool execution error → error message fed back as tool_result → Claude analyzes root cause → adjusts strategy (read file/change method/modify params) → retries 4.4 Error Message Truncation Error messages over 10K characters keep the first and last 5K: `${start}\n\n... [${length - 10000} characters truncated] ...\n\n${end}` 4.5 Turn-Level Error Tracking // Use watermark to isolate errors for each Turn: const errorLogWatermark = getInMemoryErrors().at(-1) // Turn start snapshot // ... turn execution ... const turnErrors = getInMemoryErrors().slice(watermarkIndex + 1) // only new errors Claude Code Source Deep Dive — Literal Translation (Part 5) Part V: End-to-End Query Pipeline Flow 5.1 Retry Mechanism (withRetry()) API call fails ↓ 401/403: refresh OAuth token/credentials → retry 429 (rate limited): short delay (< threshold): retry with fast mode long delay: switch to standard-speed model 529 (overload): non-foreground request: give up immediately consecutive < 3 times: exponential backoff retry consecutive ≥ 3 times: trigger model fallback Max tokens overflow: calculate available token count → adjust maxTokens → retry ECONNRESET/EPIPE: disable keep-alive → retry Persistent retry mode (UNATTENDED_RETRY): unlimited retries + exponential backoff chunked sleep + periodic status messages window rate limiting: wait until reset instead of polling 6-hour total upper bound Backoff calculation: delay = BASE_DELAY_MS × 2^(attempt-1) jitter = ±25% of base delay max = 32s (standard) / 5min (persistent) 5.2 Message Preparation Pipeline Raw messages → applyToolResultBudget() (size limit) → snipCompact() (snippet compression, feature-gated) → microCompact() (micro-compression, cache old tool_result) → contextCollapse() (phased context reduction) → autoCompact() (automatic compression, after token threshold reached) → normalizeMessagesForAPI() (API format normalization) 5.3 Streaming Tool Execution // Concurrency model Read-type tools (Grep, Glob, Read) → run in parallel, up to 10 concurrent Write-type tools (Edit, Write, Bash) → run serially, one at a time // StreamingToolExecutor states: 'queued' → 'executing' → 'completed' → 'yielded' // Interrupt handling: User interrupt → generate synthetic error messages for all queued/running tools Model fallback → discard old executor, create a new retry Sibling error → Abort sibling processes of parallel tasks 5.4 Seven Continue Points in the Query Loop collapse_drain_retry — retry after context-collapse drain reactive_compact_retry — retry after reactive compaction max_output_tokens_escalate — retry after output-token escalation max_output_tokens_

Weekly AI roundup (May 23–30, 2026): Claude Opus 4.8 Fast Mode 3x cheaper, Qwen 3.7 Max beats Claude at half the price, ChatGPT moves into Excel

Pulling together this week's major AI releases for anyone who didn't have time to track every blog post. Sticking to substantive changes, not hype. Anthropic — Claude Opus 4.8 Released this week. Headline pricing unchanged, but Fast Mode dropped from $30 input / $150 output per million tokens to $10 / $50 — a 3x reduction on the premium tier. Reported improvements in "judgment" and longer autonomous runs. Also shipped 20+ legal MCP connectors and Microsoft 365 add-ins (Excel, PowerPoint, Word) in GA. Alibaba — Qwen 3.7 Max Launched May 20 at Alibaba Cloud Summit. 1M-token context. Reported to top Claude Opus 4.6 Max on Terminal-Bench 2.0, SWE-Bench Pro, and MCP-Atlas. Pricing $2.50 / $7.50 per million tokens — roughly half of Opus 4.7. Alibaba claims autonomous operation up to 35 hours without performance degradation. Alibaba is now ranked #6 lab globally on Arena text leaderboard. OpenAI — GPT-5.5 Instant Now default in ChatGPT. Reports 52.5% fewer hallucinated claims than GPT-5.3 Instant on high-stakes prompts (medicine, law, finance). OpenAI also shipped a ChatGPT sidebar inside Excel and Google Sheets, plus a personal finance dashboard for Pro users (US only). Google — Gemini 3.5 Flash Reported to beat Gemini 3.1 Pro on coding and agentic benchmarks at ~4x faster output token rate. Ultra subscription cut from $250 to $200/month; new $100/month Developer tier introduced. xAI — Grok Build 0.1 Coding agent moved to public API beta May 28. Custom Skills feature added for reusable user-defined tasks. Connectors for SharePoint, OneDrive, Notion, GitHub, Linear, plus bring-your-own MCP support. Mistral Launched Vibe (unified work + code agent, replaces Le Chat). Acquired Emmi AI for physics-based simulation. Targeting €1B revenue in 2026; new 10MW inference DC announced. Hugging Face Launched an app store for the Reachy Mini robot. ~10,000 units shipped. Also reported a malicious repo masquerading as an OpenAI release that accumulated 244K downloads before takedown — relevant for anyone pinning models from HF in production. My take as someone building on top of these APIs: The 3x Opus Fast Mode price cut and Qwen 3.7 Max's pricing + autonomous duration are the real signal this week. The cost floor on premium-tier inference is dropping faster than most app-layer products have repriced for. Anyone running multi-step agent workflows needs to recompute unit economics this week — either pass through the savings or reinvest the margin. The other pattern worth noting: OpenAI and Anthropic are both pushing into Excel/M365 surfaces. Distribution is becoming the next battleground, not raw model capability. If you're building a productivity SaaS, the giants are now inside the same surface as you. submitted by /u/ksraj1001 [link] [comments]

I got tired of alt-tabbing between my editor and Claude Code, so I built an IDE around it — using Claude Code

For weeks my setup was three windows: editor in one, a terminal running claude in another, git in a third. I was the integration layer — copying file paths into the terminal, tabbing back to read a diff, tabbing again to stage it. The agent was great; the workflow around it was held together with muscle memory. So I built Cantus, and the fitting part is I built most of it with Claude Code. What it is: a native macOS app that gives the Claude Code CLI a real home. The actual claude CLI runs in an integrated terminal (a real PTY — sessions resume exactly like in your own terminal), next to a Monaco editor and built-in git, all sharing one window and one project. Drag a file onto the terminal and its path drops into the prompt. Diffs stage per-line, not just per-file. There's also a task runner that takes a goal, figures out which of your .claude skills and agents apply, and runs a workflow — plus a local memory layer (SQLite + FTS5, no cloud, no vector DB) that remembers a project's quirks run to run. Tauri 2 + Rust under the hood, so it's a small native binary — no Electron. How Claude Code helped build it: the fiddly Rust was the part I'd have stalled on alone — line-level git staging through libgit2's patch API, the PTY that spawns and streams claude, the typed Tauri IPC between Rust and the React frontend. I paired with Claude Code through most of it. The line-staging in particular went from "I'll get to this someday" to working in an afternoon. Free to try: open-source, MIT, no account or telemetry. brew tap manan45/cantus && brew install --cask cantus, or grab the .dmg from releases. macOS Apple Silicon for now. Repo: https://github.com/manan45/Cantus · demo + details: https://manan45.github.io/Cantus/ Happy to get into any of it — especially the choice to use FTS5 instead of a vector DB for the memory layer, which I keep expecting to regret and haven't yet. submitted by /u/Ancient-Sam2013 [link] [comments]

Claude Code Source Deep Dive (Part 5) — Literal Translation & Tool-Call Loop Self-Repair Core Mechanism

Reader’s Note On March 31, 2026, the Claude Code package Anthropic published to npm accidentally included .map files that can be reverse-engineered to recover source code. Because the source maps pointed to the original TypeScript sources, these 512,000 lines of TypeScript finally put everything on the table: how a top-tier AI coding agent organizes context, calls tools, manages multiple agents, and even hides easter eggs. I read the source from the entrypoint all the way through prompts, the task system, the tool layer, and hidden features. I will continue to deconstruct the codebase and provide in-depth analysis of the engineering architecture behind Claude Code. 3.14 EnterWorktree Tool (Enter Worktree) Create isolated git worktree and switch current session into it. When to Use: - User explicitly says "worktree" When NOT to Use: - User asks to create/switch branches - User asks to fix bug or work on feature without mentioning worktrees - NEVER use unless user explicitly mentions "worktree" Behavior: - Creates new git worktree inside `.claude/worktrees/` with new branch - Switches session's working directory to new worktree 3.15 AskUserQuestion Tool (Ask User Question) Ask user multiple choice questions to gather info, clarify ambiguity, understand preferences, make decisions, offer choices. Usage Notes: - Users always able to select "Other" for custom text input - Use multiSelect: true to allow multiple answers - If recommend specific option, make first option with "(Recommended)" at end Preview Feature: - Use optional `preview` field on options when presenting concrete artifacts needing visual comparison (ASCII/HTML mockups, code snippets, diagrams) - Preview content rendered as monospace markdown - When any option has preview, UI switches to side-by-side layout 3.16 LSP Tool (Language Server) Interact with Language Server Protocol servers for code intelligence. Supported Operations: - goToDefinition, findReferences, hover, documentSymbol, workspaceSymbol, goToImplementation, prepareCallHierarchy, incomingCalls, outgoingCalls All Operations Require: - filePath, line (1-based), character (1-based) 3.17 Sleep Tool (Wait) Wait for specified duration. Usage: - When user tells to sleep/rest - When nothing to do / waiting for something - May receive periodic check-ins (tick tags) - Can call concurrently with other tools - Prefer over `Bash(sleep ...)` — doesn't hold shell process - Each wake-up costs API call - Prompt cache expires after 5 min inactivity 3.18 CronCreate Tool (Scheduled Task) Schedule prompts to run at future times. Uses standard 5-field cron in user's local timezone. One-Shot Tasks (recurring: false): - "remind me at X" → pin minute/hour/day to specific values Recurring Jobs (recurring: true, default): - "every 5 min" → "*/5 * * * *" - "hourly" → "0 * * * *" CRITICAL: Avoid :00 and :30 Minute Marks (when task allows) - Every user asking "9am" gets 0 9, causing thundering herd - When approximate: pick minute NOT 0 or 30 - "every morning around 9" → "57 8 * * *" (not "0 9 * * *") Durability: - Default (durable: false): lives only in Claude session - durable: true: writes to .claude/scheduled_tasks.json Recurring tasks auto-expire after 7 days. 3.19 TeamCreate Tool (Create Team) Create team to coordinate multiple agents working on project. When to Use (Proactively): - User explicitly asks to use team, swarm, or group agents - Task complex enough for parallel work Team Workflow: 1. Create team with TeamCreate 2. Create tasks using Task tools 3. Spawn teammates using Agent tool with team_name + name params 4. Assign tasks using TaskUpdate with owner 5. Teammates work on assigned tasks 6. Shutdown gracefully via SendMessage with shutdown_request IMPORTANT: Always refer to teammates by NAME. Plain text output NOT visible to other agents — MUST call SendMessage tool to communicate. 3.20 ToolSearch Tool (Deferred Tool Search) Fetch full schema definitions for deferred tools so they can be called. Query Forms: - "select:Read,Edit,Grep" — fetch exact tools by name - "notebook jupyter" — keyword search, up to max_results best matches - "+slack send" — require "slack" in name, rank by remaining terms submitted by /u/Ill-Leopard-6559 [link] [comments]

Ai Benchmarks are useless

I'm done with the launch cycle. Every new model drops with the same flashy report, bar charts all over the place, hitting 92% on MMLU-Pro, 94% on GPQA, or whatever coding benchmark they're pushing this week. Then you plug it into a real workflow through the API, or try to run it on an actual multi-step project that's not some tidy puzzle, and it feels like a step back from what we had a year ago. This is Goodhart’s Law playing out completely. The labs tuned everything for the tests, and now we've got these fragile models that break down in production. The benchmarks themselves are mostly cooked at this point. The ones they still brag about are saturated or contaminated. Classic MMLU and HumanEval don't tell you much anymore for frontier models. Scores are all bunched up in the high 80s to low 90s, so a couple points difference is basically noise. It doesn't mean one is actually smarter. On top of that, these tests have been public forever. Training data and synthetic stuff pick them up, so the model isn't really reasoning through new problems. It's pattern matching from stuff it saw during training. Move to fresher setups like LiveBench or real agent workflows and the numbers drop hard. They also gloss over the harness they use for those record scores. Heavy scaffolding, multi-shot prompts tuned exactly to the eval, extra compute with internal loops and all that. In real work you just send normal prompts. Take that away and the performance evaporates. Suddenly it can't hold basic JSON output without babying it. Tweak a few words in the prompt and your results swing 10-20 points. What actually feels worse day to day is stuff like this: the big context windows sound great on paper but retrieval in the middle is weak, it drops instructions a few turns in, or fails to pull details across documents properly. On coding, it might patch one isolated GitHub issue okay, but drop it in a real messy codebase and it starts making up library methods that don't exist, quits halfway, or leaves TODO placeholders where the actual logic needs to go. Reasoning turns into these long pedantic loops even for straightforward tasks instead of just getting it done. And the safety layer is twitchy enough that normal business words like execute or termination make it refuse to touch a spreadsheet. We're way past the point where a higher benchmark score means a better daily tool. The incentives push models to ace closed tests while making them less flexible, more wordy, and annoying to integrate. Until things shift to fresh dynamic evals and real human preference in messy conditions, most of these announcements are marketing wins more than anything else. submitted by /u/Significant-Care-135 [link] [comments]

Why do we have visual programming for code, but not for prompts?

Prompt Logic Gates (PLG) GitHub Repository Something I've been thinking about recently. In software development, we've spent decades building abstractions to make complex systems manageable: Functions instead of repeating code Classes and modules instead of giant files Visual systems such as Unreal Blueprints, Node-RED, and LabVIEW. Compilers that validate and transform input before execution But when it comes to AI prompts, many of us are still writing massive text blobs. A complex prompt can easily become hundreds of words long with multiple responsibilities: Context Constraints Style instructions Exclusions Decision logic Fallback behavior At that point, it starts feeling less like text and more like a program. That made me wonder: Why don't we treat prompts as executable logic? Imagine building prompts using logic gates: AND → merge instructions OR → choose between alternatives NOT → remove unwanted concepts Question nodes → identify missing requirements Compiler → validate contradictions before execution Instead of editing a giant string, you'd build a graph and compile it into the final prompt. I've been experimenting with this idea in a prototype called Prompt Logic Gates (PLG). It treats prompts like compilable programs, using concepts such as dependency graphs, execution order, semantic conflict detection, visual nodes, and compilation pipelines. such as Unreal Blueprints, Node-RED, and LabVIEW Repo: Prompt Logic Gates (PLG) GitHub Repository I'm not posting this as a product launch or anything — I'm more interested in whether this direction makes sense from a software engineering perspective. Do you think prompts eventually become a programming layer of their own? Or will natural language always be the better abstraction? Curious what other developers think. submitted by /u/withsj [link] [comments]

After months of "better prompts," what actually 10x'd my Claude Code was treating it like an OS, not a chatbot

Spent way too long collecting prompts thinking that was the bottleneck. It wasn't. The shift that worked: Claude Code has five layers and most of us only use one (the message box). The other four — CLAUDE.md, skills, hooks, subagents — are where the leverage is. The single biggest win was a ~30-line CLAUDE.md at the repo root. Standing rules the agent reads every session. Stopped re-explaining my project daily, stopped it reaching for the library we'd banned, tests started running on their own. Wrote up the full breakdown (the five layers, the CLAUDE.md, the skills, the subagent setup) here if useful: https://medium.com/p/6882e77f0b65?postPublishedType=initial Curious what's in other people's CLAUDE.md — what rules made the biggest difference for you? submitted by /u/DeepThroatStroky [link] [comments]

Karpathy LLM OS Layer

┌──────────────────────────────────────────────────────────────────────────┐ │ Karpathy LLM OS Layer │ │ LLM=CPU │ Context=RAM │ Storage=Disk │ Tools=System Calls │ │ Skills=Programs │ Harness=Kernel │ Agent Teams=Processes │ │ ┌──────────────────────────────────────────────────────────────────┐ │ │ │ context-manager: Token Budget → Prompt Assembly → Truncation │ │ │ │ token-cost-tracker: Estimate → Log → Report │ │ │ └──────────────────────────────────────────────────────────────────┘ │ └──────────────────────────────────────────────────────────────────────────┘ │ ┌──────────┴──────────┐ ▼ ▼ ┌──────────────────┐ ┌──────────────────────┐ │ External │ │ Agent Teams │ │ Sources │ │ (Parallel Fleet) │ └────────┬─────────┘ └──────────────────────┘ ▼ ┌──────────────────────────────┐ │ wiki-ingest + knowledge-ops│ │ (STOW pipeline + RAG sync) │ └──────┬──────────┬────────────┘ │ │ ┌──────▼ └──────────────┐ │ Knowledge Layers │ │ ├ Active (GitHub/Linear) │ │ ├ Memory (quick access) │ │ ├ Wiki (durable, interlinked) │ │ ├ Vector (ChromaDB, semantic) │ │ └ External (DBs, APIs) │ └────────────────────────────────┘ │ ┌───────────┼──────────┬──────────────┬──────────────┐ ▼ ▼ ▼ ▼ ▼ ┌─────────┐ ┌─────────┐ ┌──────────┐ ┌───────────┐ ┌──────────┐ │ daily │ │cognitive│ │ behavior │ │ creativity│ │ project │ │ -okr │ │-compile │ │ -design │ │ -engine │ │ -flow-ops│ └─────────┘ └─────────┘ └──────────┘ └───────────┘ └──────────┘ │ │ │ │ │ └───────────┼──────────┼──────────────┼──────────────┘ ▼ ┌─────────────────────────────────────────────────────────────┐ │ session-learn (+Closure Protocol) ← feedback loop │ │ verify-before-claim ← quality gate │ │ wiki-lint ← health check │ │ deep-research ← synthesis │ │ harness-engineering ← safety + multi-agent │ │ agent-teams-command ← fleet command │ │ startup-evaluation ← VC evaluation │ │ anthropic-os ← work method engine │ └─────────────────────────────────────────────────────────────┘ submitted by /u/Master_Ear_2984 [link] [comments]

Blaming the model won't fix your workflow — a white paper on structural enforcement for AI agents

I've been working on something others might find interesting. It's under heavy development as I learn. Most AI agent setups treat the model like a better autocomplete — paste a prompt, get output, hope it's right. That works for small tasks. It falls apart when you try to use agents for sustained work across sessions: they skim specs, declare victory at 60%, burn context on noise, silently resolve ambiguity without surfacing it, and mark checklist items done without actually doing them. The failures are predictable and nameable — so I named them. This is a white paper and implementation guide for a full-stack agentic system — everything from planning through promotion under structural enforcement. It documents 24 failure modes from months of multi-agent operation and, for each, describes what actually prevents it: some through mechanical gates the agent cannot skip, some through procedural skills, and some through human supervision. The guide covers how to structure specs, plans, and verification so that agent work is evidence-led rather than vibes-led, how to use MCP capability surfaces as structural levers, and how the failure modes apply regardless of which model or vendor you use. The white paper also includes a Related Work section that positions it against the emerging industry consensus — CodeRabbit, Anthropic, Spotify, Cloudflare, OpenAI, Karpathy, Thoughtworks, and academic research all independently arrived at pieces of the same conclusions. The difference here is the integrated stack: a failure taxonomy mapped to prevention mechanisms, a three-layer enforcement architecture, and a concrete reference implementation with an orchestrator, task graphs, step verification, adversarial review, and model stratification. White paper: https://gitlab.com/naive-x/naive-artifact-coding/-/blob/main/white-paper.md Reference implementation: https://gitlab.com/naive-x/naive-artifact-coding/-/blob/main/docs/reference-implementation-guide.md Implementation guide: https://gitlab.com/naive-x/naive-artifact-coding/-/blob/main/implementation-guide.md The methodology is language-agnostic. The reference implementation is in Common Lisp, but the architecture (orchestrator, supervisor, MCP servers, task graphs, event emission) doesn't assume any particular language or domain. There are companion specs for adapting it to enterprise workflows. submitted by /u/Harag [link] [comments]