Up-and-Down and the Percentile-Finding Problem

Up-and-Down (U&D) is a popular sequential design for estimating threshold percentiles in binary experiments. However, U&D application practices have stagnated, and significant gaps in understanding its properties persist. The first part of my work aims to fill gaps in U&D theory. New results concerning stationary distribution properties are proven. A second focus of this study is nonparametric U&D estimation. An improvement to isotonic regression called “centered isotonic regression” (CIR), and a new averaging estimator called “auto-detect” are introduced and their properties studied. Bayesian percentile-finding designs, most notably the continual reassessment method (CRM) developed for Phase I clinical trials, are also studied. In general, CRM convergence depends upon random run-time conditions – meaning that convergence is not always assured. Small-sample behavior is studied as well. It is shown that CRM is quite sensitive to outlier sub-sequences of thresholds, resulting in highly variable small-sample behavior between runs under identical conditions. Nonparametric CRM variants exhibit a similar sensitivity. Ideas to combine the advantages of U&D and Bayesian designs are examined. A new approach is developed, using a hybrid framework, that evaluates the evidence for overriding the U&D allocation with a Bayesian one.

💡 Research Summary

This dissertation addresses the long‑standing problem of estimating a single target percentile (Qₚ) in binary response experiments where sample sizes are small, the underlying threshold distribution is unknown, and treatment levels are confined to a discrete set. The author surveys two dominant families of sequential designs—Up‑and‑Down (U&D) methods, which generate a Markov chain of treatment allocations, and Bayesian designs, most prominently the Continual Reassessment Method (CRM) used in Phase I clinical trials.
In the first part, the thesis develops new theoretical results for U&D designs. All variants are shown to induce a Markov chain, and the stationary distribution of the “k‑in‑a‑row” (KR) design is derived rigorously. Contrary to some recent claims, KR possesses a single‑mode stationary distribution, and the author proves that KR dominates other U&D variants in both convergence rate and asymptotic estimation precision for a given target percentile. This fills a notable gap in the literature that had left the long‑run behavior of many U&D schemes poorly understood.
The second part focuses on non‑parametric estimation within the U&D framework. Traditional isotonic regression (IR) is recognized for its monotonicity enforcement but suffers from boundary bias. The author introduces Centered Isotonic Regression (CIR), which recenters each isotonic estimate to the midpoint of its supporting interval, thereby reducing bias and improving interval‑estimation coverage. A complementary averaging estimator, called “auto‑detect,” automatically identifies the transition from the early exploratory phase (characterized by frequent reversals) to the stable phase and computes separate averages for each, yielding lower mean‑squared error. Both CIR and auto‑detect are accompanied by theoretical bias‑variance analyses, bootstrap confidence‑interval procedures, and extensive simulation studies that demonstrate superior performance over existing U&D estimators across normal, exponential, and mixed threshold distributions. An applied anesthesiology experiment illustrates the practical gains.
The third part scrutinizes Bayesian percentile‑finding designs, especially CRM. While CRM is widely believed to converge to the optimal dose, the dissertation proves that convergence is contingent on random run‑time conditions; it is not guaranteed in every realization. The analysis reveals that outlier subsequences of thresholds can dramatically perturb CRM’s allocation rule, leading to highly variable outcomes even under identical design settings. Non‑parametric CRM variants exhibit the same sensitivity, making small‑sample behavior unpredictable. The author derives sufficient conditions for convergence (e.g., appropriate prior concentration, bounded escalation steps) and quantifies the impact of outliers on posterior updates.
Building on these insights, the author proposes a hybrid framework named Bayesian Up‑and‑Down (BUD). The core idea is to follow the U&D allocation as long as the accumulated evidence supports it; when the posterior distribution indicates substantial uncertainty—measured via Kullback‑Leibler divergence, Bayesian Information Criterion thresholds, or posterior probability bounds—the design switches to a Bayesian allocation for the remainder of the trial. Simulation experiments show that BUD retains the robustness of U&D while achieving the efficiency of Bayesian methods: it exhibits lower variance than CRM, higher precision than pure U&D, and markedly reduced sensitivity to outlier threshold sequences. The hybrid also performs well across a range of underlying distributions and sample sizes.
The dissertation concludes by summarizing the contributions: (1) rigorous stationary‑distribution theory for U&D, especially KR; (2) novel non‑parametric estimators (CIR and auto‑detect) with proven bias‑variance properties; (3) a nuanced understanding of CRM’s convergence and its vulnerability to small‑sample anomalies; and (4) a practical, data‑driven hybrid design (BUD) that leverages the strengths of both paradigms. Future work is outlined, including extensions to multivariate treatment spaces, simultaneous estimation of multiple percentiles, and adaptive prior updating in real time. Overall, the work provides a comprehensive, theoretically grounded, and practically applicable roadmap for percentile‑finding experiments in biomedical, psychophysical, and industrial settings.