Abstract
Item difficulty prediction from item content has been studied using linguistic features, supervised models, and large language models (LLMs), but many approaches ask models to produce calibrated numeric difficulty values directly. This study proposes a Bookmark-style placement framework in which an LLM places a new item relative to calibrated anchor items in systematically interleaved booklets. Per-booklet placements are converted to known p values and combined as panelist-style estimates. The framework was evaluated with Claude Opus 4.6, GPT-5.4, and GPT-5.5 under ascending-order, descending-order, and double-pass conditions across multiple booklet lengths and two item pools: grade 7 Mathematics and Verbal Reasoning. The results showed meaningful prediction signal across models and item types, systematic prompt-order bias on the known p scale, and stronger performance in Mathematics than in Verbal Reasoning. Findings suggest that LLM-based difficulty prediction can be productively framed as ordinal placement rather than direct continuous estimation, while booklet design, prompt order, item domain, and downstream calibration all affect operational usefulness.