Questions before papers
Start with the decisions and uncertainties that matter. Papers, evaluations, data, and models then become evidence attached to those questions.
1. The proposal
A small, persistent map—not another large database
Each group could maintain a limited set of questions and research clusters that seem especially likely to inform consequential decisions. Individual members could keep their own suggestions; the group view would summarize the discussion without implying agreement where there isn't any.
Keep disagreement visible
The list needn't be a consensus ranking. Members can give separate judgments, propose alternatives, and explain why they would prioritize something differently.
Maintain, prune, and revisit
Each active item would have a human owner and a review date. Deprioritized items would remain visible with a short explanation, rather than quietly disappearing.
What this would not replace
This is not intended to replace discussion of actual papers, prioritization assessments, final voting, evaluation management, or questions about our process. It would add a more durable layer connecting those activities to the larger research questions and decisions we care about.
2. A possible structure
One compact table, with several views
The same underlying material could support a group shortlist, individual members' suggestions, prompts imported from existing prioritization work, and a record of retired items.
| Question or cluster | Who might use the answer? | Why it may matter | Current disagreement | Promising next work | Owner and review |
|---|---|---|---|---|---|
| Short, decision-relevant formulation | Named types of funders, policymakers, researchers, or advocates | A concise path from better evidence to better decisions | What informed people disagree about, including minority views | Evaluation, synthesis, replication, modeling, new data, or experiment | Human owner Review every 6–12 months |
Suggested views
- Group-curated: perhaps five to ten active questions or clusters.
- Member suggestions: separate nominations and judgments, not forced into a common ranking.
- Prompts to consider: items drawn from our databases, funder agendas, crux mapping, or AI-assisted research.
- Deprioritized or retired: preserved with reasons and a date.
Possible lightweight judgments
- How much could an answer change real decisions?
- How uncertain or disputed is the answer?
- Is useful progress tractable in the next few years?
- Is the area receiving attention roughly proportionate to its importance?
- What evidence would most change the member's view?
3. Illustrative starting points
Examples to react to—not proposed group conclusions
These draw loosely on existing Unjournal prioritization work and Pivotal Questions projects. They are deliberately incomplete.
Animal welfare, markets, and attitudes
- When do plant-based products reduce purchases of high-welfare-cost animal products, rather than adding variety or replacing less harmful products?
- Which assumptions drive disagreement over whether cultivated meat can become cost-competitive?
- Which animal-welfare measures are credible enough for comparing interventions, species, and population changes?
Global health and development
- Which constraints most limit the scale-up of otherwise promising interventions?
- When should funders generalize from an RCT, and when will implementation at scale produce substantially different results?
- Which major funder or NGO conclusions depend on contestable empirical assumptions that merit independent evaluation?
AI governance and catastrophic risks
- Which empirical claims about AI scaling are decision-relevant and uncertain enough to merit independent technical auditing?
- How informative are current dangerous-capability evaluations for real deployment risks and governance choices?
- Which governance proposals depend on empirical assumptions receiving less scrutiny than the legal or philosophical arguments?
Wellbeing, psychology, and welfare economics
- Under what assumptions are subjective-wellbeing changes comparable across people, time, and contexts?
- When, if ever, can WELLBYs be converted to QALYs or DALYs without concealing important normative choices?
- Which bounded null results or meta-analyses could lead funders to stop, redesign, or expand major programs?
4. Human judgment and AI
AI can supply prompts, but it shouldn't become the group's view
AI research prioritization, literature mapping, and crux mapping may be useful inputs. The group should retain authorship, judgment, and control.
Independent input first
Members could record their own candidate questions before seeing AI-generated suggestions, reducing the risk that the prepared material anchors the whole discussion.
Clear provenance
AI-generated and database-imported suggestions should appear in a separate, labeled view, with the source and date. They are prompts to criticize, combine, rewrite, or discard.
Human responsibility
A named person should rewrite and take responsibility for anything promoted to the curated list. AI should never overwrite a member's judgment or silently revise the record.
5. A flexible meeting format
This needn't take over the field specialist meeting
The mix should vary with what is timely. If several strong papers need decisions, paper discussion may take most of the meeting. At other meetings, the group might spend longer on the broader priority map.
0–5 min
Updates and agenda choices
Agree on where the meeting's attention is most useful today.
5–25 min
Actual research papers
Discuss promising papers, prioritization assessments, final votes, and possible evaluation managers.
25–35 min
Process questions
Clarify how research moves through the pipeline and identify avoidable frictions or gaps.
35–50 min
Pivotal questions and directions
Add, challenge, refine, or retire items in the group's longer-term priority map.
50–60 min
Decisions and ownership
Record next steps, owners, unresolved disagreements, and what should return at a later meeting.
Optional +20
Members' research and research use
An informal space for people who can stay longer to discuss their own work, how research is being used, possible collaborations, or useful feedback.
This is one possible allocation, not a standard agenda. The group could shorten, extend, reorder, or omit sections. Some meetings might be almost entirely about papers; an occasional meeting might focus mainly on the broader priority map.
6. Questions for Monday
What should we keep, change, or drop?
About the underlying idea
- Would a maintained question-and-cluster map actually help your group?
- Should there be a short group-curated list, individual lists, or both?
- Who might use the result—and what would make it genuinely useful to them?
- How often should it be revisited?
About the meeting and workflow
- How much meeting time, if any, should this receive?
- Should independent member suggestions come before prepared database and AI prompts?
- What is the minimum information needed for an item to enter the curated list?
- Would anyone be willing to help pilot or maintain this for one group?
A possible small pilot
Try this with two or three active, decision-linked groups. Ask members for independent suggestions, use existing Unjournal prioritization material as prompts, and leave each group with no more than five to ten active items. Revisit the experiment after one or two meetings before building a more elaborate Coda system.
Why start with a hosted page?
This lets us discuss the concept without adding another permanent Coda structure first. If the framework seems useful, the parts worth maintaining can later move into a compact Coda table or connect to the existing research-prioritization database.