ForkFree

Prompts / Research and analysis

Reconstruct How a Paper Was Thought Up

You get a critical reading that explains why the method looks the way it does, where it would break, and what the paper leaves out. Forcing a failure story for each mechanism and an evidence tag for each claim stops the reply from just repeating the abstract.

Partly workedIt has the five sections, the three questions and mostly sound sceptical reasoning. But it mislabels absences as [stated], and the worked example ends with a muddled 0.2%-to-0.1% jump instead of a whole-model count, so the reader can't check the headline number. Some jargon is also left unexplained for a non-ML reader.

The prompt

Act as a sceptical senior researcher who reads papers the way a lab lead would, looking for the decisions behind the polished write-up.

Task: Read the paper text below and reconstruct how the authors probably arrived at their method, rather than summarising the abstract.

Paper text: {paper_text}
Reader background: {reader_background}

Steps:
1. Describe the concrete situation where earlier methods break, and the bet the authors made in response.
2. For each key mechanism, explain how a first, naive version would fail, the cheapest check a lead would run to see it, and the patch with its cost.
3. Pick the most important formula or number and work it through with a tiny example the reader can check by hand.
4. Note what the paper does not report, and where the wording is confident or evasive.
5. Tag every claim as [stated], [inferred] or [guessed].

Output: Five short headed sections matching the steps, in plain English pitched at the reader's background, followed by a three-item list of questions to ask the authors.

Change the parts in curly braces. Use it when you need to decide whether to trust or adopt a method from a paper and the abstract feels too tidy.

What happened when we ran it

We sent

Act as a sceptical senior researcher who reads papers the way a lab lead would, looking for the decisions behind the polished write-up.

Task: Read the paper text below and reconstruct how the authors probably arrived at their method, rather than summarising the abstract.

Paper text: We propose LoRA-Lite, which freezes the pretrained weights and trains a rank-4 update added to the attention projections. On a 7B model, it matches full fine-tuning on three benchmarks while training 0.1% of the parameters. We set the scaling factor alpha to 32 and use a learning rate of 3e-4. Training uses a single seed. We did not evaluate on long-context tasks.
Reader background: I run a small software agency. I can code in Python and understand basic matrices, but I have no machine learning research training.

Steps:
1. Describe the concrete situation where earlier methods break, and the bet the authors made in response.
2. For each key mechanism, explain how a first, naive version would fail, the cheapest check a lead would run to see it, and the patch with its cost.
3. Pick the most important formula or number and work it through with a tiny example the reader can check by hand.
4. Note what the paper does not report, and where the wording is confident or evasive.
5. Tag every claim as [stated], [inferred] or [guessed].

Output: Five short headed sections matching the steps, in plain English pitched at the reader's background, followed by a three-item list of questions to ask the authors.

The model replied

# Reading "LoRA-Lite" like a lab lead Note: the paper text is only a few sentences, so much of what follows is reconstruction. I tag each claim as [stated], [inferred] or [guessed]. ## 1. Where earlier methods break, and the bet **The situation.** Full fine-tuning of a 7B model updates every weight. That means storing gradients and optimizer state for all ~7 billion numbers, which needs several times the model's own memory. [inferred] A small team can't afford this, and keeping one full copy of the model per task is also costly. [inferred] The paper itself doesn't describe this problem. [stated: it describes no motivation in the text given] **The bet.** Most of what fine-tuning changes lives in a low-dimensional subspace, so you can freeze the original weights and learn a tiny correction. [inferred] Concretely: add a rank-4 update to the attention projections and train only that. [stated] The claimed payoff is matching full fine-tuning on three benchmarks with 0.1% of the parameters. [stated] **What's not new.** This is essentially standard LoRA (low-rank adaptation). The "Lite" label suggests a variant, but the text names no difference from LoRA. [inferred] A lab lead's first question is: what is actually novel here? ## 2. Key mechanisms: naive failure, cheap check, patch **Mechanism A: low-rank update (rank 4).** - *Naive failure:* Rank 4 may be too small to express what the task needs, so the model underfits on harder tasks. [inferred] - *Cheapest check:* Train at ranks 1, 4, 16, 64 and plot the score. If the curve is flat, rank barely matters. If 4 is on a steep part, the "matches full fine-tuning" claim is fragile. [inferred] - *Patch and cost:* Raise the rank. Trainable parameters grow linearly with rank, which weakens the "0.1%" headline. [inferred] **Mechanism B: only the attention projections get updates.** - *Naive failure:* Knowledge or behaviour that lives in the feed-forward layers can't be changed. [guessed] - *Cheapest check:* Run the same setup with the feed-forward layers also adapted and see whether scores move. [inferred] - *Patch and cost:* Adapt more layers, which costs more parameters and memory. [inferred] **Mechanism C: scaling factor alpha = 32 and learning rate 3e-4.** - *Naive failure:* In LoRA the update is typically multiplied by alpha/rank. With rank 4 and alpha 32, that multiplier is 8, which is a strong amplification. Combined with the learning rate, results could be sensitive to these choices. [inferred] The paper gives no sign these were tuned, or tuned equally for the baseline. [stated: no tuning described] - *Cheapest check:* Sweep learning rate (say 1e-4, 3e-4, 1e-3) for both LoRA-Lite and full fine-tuning. A common trick is giving the proposed method a tuned learning rate and the baseline a default one. [guessed that this happened here] - *Patch and cost:* Tune both fairly. Cost is several more training runs per method. **Mechanism D: single seed.** - *Naive failure:* Differences of a point or so between runs can come from random initialisation and data order alone. "Matches" could be noise. [inferred] - *Cheapest check:* Rerun with 3 to 5 seeds and report mean and spread. [inferred] - *Patch and cost:* Multiply training compute by the number of seeds. For a small adapter, this is cheap. [inferred] ## 3. Worked example: parameter count and the alpha scaling The 0.1% claim comes from this arithmetic. Take one weight matrix of size d × d. Full fine-tuning trains d² numbers. A rank-r update trains two small matrices, A (d × r) and B (r × d), so 2·d·r numbers. The learned change is the product B·A, scaled by alpha/r. [inferred, standard LoRA form; the paper's exact formula isn't given] **Tiny example you can check by hand:** let d = 4096 (typical for a 7B model [guessed]) and r = 4. - Full: 4096 × 4096 = 16,777,216. - Low-rank: 2 × 4096 × 4 = 32,768. - Ratio: 32,768 / 16,777,216 ≈ 0.2% for that one matrix. So adapting only some matrices lands near the claimed 0.1% overall once you count the rest of the model. [inferred] The number is plausible, though I can't verify which matrices were adapted. Scaling: alpha/r = 32/4 = 8, so the learned update is multiplied by 8 before being added to the frozen weights. If B·A has an entry of 0.01, the added change is 0.08. [inferred] ## 4. What the paper doesn't report, and tone **Missing:** - Which three benchmarks, which baseline settings, and the actual scores. [stated: benchmarks unnamed] - What "matches" means: within a point, within noise, or statistically? [stated: undefined] - Variance across seeds. Only one seed was used. [stated] - Long-context performance. [stated: not evaluated] - Memory use, training time, and any comparison to ordinary LoRA or other adapters. [inferred from absence] - How hyperparameters were chosen. [inferred from absence] - Whether other model sizes were tried. Only 7B appears. [stated] **Tone:** - *Confident:* "matches full fine-tuning" is a strong, unqualified claim resting on one seed and three benchmarks. [inferred] - *Honest but buried:* "We did not evaluate on long-context tasks" is a real disclosure, but it's one flat sentence with no discussion of what it implies. [inferred] - *Evasive by omission:* "Training uses a single seed" is stated as a fact, not flagged as a limitation. [inferred] ## Questions for the authors 1. How does LoRA-Lite differ from standard LoRA, and how does it compare against it at the same rank and learning rate? 2. Were the learning rate and alpha tuned for the full fine-tuning baseline as carefully as for your method, and what do results look like across at least three seeds? 3. Which three benchmarks, and by how much do you "match" full fine-tuning on each? Does the gap change at ranks other than 4?

Run on Claude Sonnet 5.5, 08 Nov 2026. Automatic check: 3 out of 5.

Mentioned for

Claude, ChatGPT, Gemini

Where it came from

Found on GitHub, shared by Wang-auspicious: Wang-auspicious/paper-xray.

This is our own version of the idea, written and tested by us. The original may differ.

More research and analysis prompts

Research and analysis Worked in our test
Skeptic Review Against Pass/Fail Criteria

You get a verdict of PASS, FAIL or UNCERTAIN on each requirement, with quoted evidence, so you know exactly what to fix. It works because the criteria are agreed up front and the reviewer is told to find faults rather than be agreeable.

Found on Web, shared by The Neuron (via Don Giannatti's newsletter)
Research and analysis Worked in our test
Evidence Ledger Before You Calculate

You get an answer where every figure can be traced back to a numbered fact, with the sums shown line by line. Labelling evidence first and calculating separately stops the model guessing numbers or blending facts together.

Found on GitHub, shared by uclanlp
Research and analysis Worked in our test
Screen Documents for Hidden AI Instructions

You get a flagged list of any text trying to steer the AI, plus an honest analysis of the document and a simple risk rating. It works because the AI is told up front that the document is data, and it must report instructions before doing anything else.

Found on Hacker News, shared by doodlesdev