Yajun Mei
Yajun Mei
Scroll
Professor of Biostatistics
-
Professional overview
-
Yajun Mei is a Professor of Biostatistics at NYU/GPH, starting from July 1, 2024. He received the B.S. degree in Mathematics from Peking University, Beijing, China, in 1996, and the Ph.D. degree in Mathematics with a minor in Electrical Engineering from the California Institute of Technology, Pasadena, CA, USA, in 2003. He was a Postdoc in Biostatistics in the renowned Fred Hutch Cancer Center in Seattle, WA during 2003 and 2005. Prior to joining NYU, Dr. Mei was an Assistant/Associate/Full Professor in H. Milton Stewart School of Industrial and Systems Engineering at the Georgia Institute of Technology, Atlanta, GA for 18 years from 2006 to 2024, and had been a co-director of Biostatistics, Epidemiology, and Study Design (BERD) of Georgia CTSA since 2018.
Dr. Mei’s research interests are statistics, machine learning, and data science, and their applications in biomedical science and public health, particularly, streaming data analysis, sequential decision/design, change-point problems, precision/personalized medicine, hot-spots detection for infectious diseases, longitudinal data analysis, bioinformatics, and clinical trials. His work has received several recognitions including Abraham Wald Prizes in Sequential Analysis in both 2009 and 2024, NSF CAREER Award in 2010, an elected Fellow of American Statistical Association (ASA) in 2023, and multiple best paper awards.
-
Education
-
BS, Mathematics, Peking UniversityPhD, Mathematics, California Institute of Technology
-
Honors and awards
-
Fellow of American Statistical Association (2023)Star Research Achievement Award, 2021 Virtual Critical Care Congress (2021)Best Paper Competition Award, Quality, Statistics & Reliability of INFORMS (2020)Bronze Snapshot Award, Society of Critical Care Medicine (2019)NSF Career AwardThank a Teacher Certificate, Center for Teaching and Learning (2011201220162020202120222023)Abraham Wald Prize (2009)Best Paper Award, 11th International Conference on Information Fusion (2008)New Researcher Fellow, Statistical and Applied Mathematical Sciences Institute (2005)Fred Hutchinson SPAC Travel Award to attend 2005 Joint Statistical Meetings, Minneapolis, MN (2005)Travel Award to 8th New Researchers Conference, Minneapolis, MN (2005)Travel Award to IEEE International Symposium on Information Theory, Chicago, IL (2004)Travel Award to IPAM workshop on inverse problem, UCLA, Los Angeles, CA (2003)Fred Hutchinson SPAC Course Scholarship (2003)Travel Award to the SAMSI workshop on inverse problem, Research Triangular Park, NC (2002)
-
Publications
Publications
Quickest change detection and Kullback-Leibler divergence for two-state hidden Markov models
AbstractAbstractThe quickest change detection problem is studied in two-state hidden Markov models (HMM), where the vector parameter θ of the HMM may change from θ0 to θ1 at some unknown time, and one wants to detect the true change as quickly as possible while controlling the false alarm rate. It turns out that the generalized likelihood ratio (GLR) scheme, while theoretically straightforward, is generally computationally infeasible for the HMM. To develop efficient but computationally simple schemes for the HMM, we first show that the recursive CUSUM scheme proposed in Fuh (Ann. Statist., 2003) can be regarded as a quasi-GLR scheme for some suitable pseudo post-change hypotheses. Next, we extend the quasi-GLR idea to propose recursive score schemes in a more complicated scenario when the post-change parameter θ1 of the HMM involves a real-valued nuisance parameter. Finally, our research provides an alternative approach that can numerically compute the Kullback-Leibler (KL) divergence of two-state HMMs via the invariant probability measure and the Fredholm integral equation.Quickest Change Detection and Kullback-Leibler Divergence for Two-State Hidden Markov Models
AbstractFuh, C. D., & Mei, Y. (n.d.).Publication year
2015Journal title
IEEE Transactions on Signal ProcessingVolume
63Issue
18Page(s)
4866-4878AbstractIn this paper, the quickest change detection problem is studied in two-state hidden Markov models (HMM), where the vector parameter θ of the HMM changes from θ0 to θ1 at some unknown time, and one wants to detect the true change as quickly as possible while controlling the false alarm rate. It turns out that the generalized likelihood ratio (GLR) scheme, while theoretically straightforward, is generally computationally infeasible for the HMM. To develop efficient but computationally simple schemes for the HMM, we first discuss a subtlety in the recursive form of the generalized likelihood ratio (GLR) scheme for the HMM. Then we show that the recursive CUSUM scheme proposed in Fuh (Ann. Statist., 2003) can be regarded as a quasi-GLR scheme for pseudo post-change hypotheses with certain dependence structure between pre- and postchange observations. Next, we extend the quasi-GLR idea to propose recursive score schemes in the scenario when the postchange parameter θ1 of the HMM involves a real-valued nuisance parameter. Finally, the Kullback-Leibler (KL) divergence plays an essential role in the quickest change detection problem and many other fields, however it is rather challenging to numerically compute it in HMMs. Here we develop a non-Monte Carlo method that computes the KL divergence of two-state HMMs via the underlying invariant probability measure, which is characterized by the Fredholm integral equation. Numerical study demonstrates an unusual property of the KL divergence for HMM that implies the severe effects of misspecifying the postchange parameter for the HMM.Comment on "Quantifying long-term scientific impact"
AbstractWang, J., Mei, Y., & Hicks, D. (n.d.).Publication year
2014Journal title
ScienceVolume
345Issue
6193Page(s)
149bAbstractWang et al. (Reports, 4 October 2013, p. 127) claimed high prediction power for their model of citation dynamics. We replicate their analysis but find discouraging results: 14.75% papers are estimated with unreasonably large μ (>5) and λ (>10) and correspondingly enormous prediction errors. The prediction power is even worse than simply using short-term citations to approximate long-term citations.Online parallel monitoring via hard-thresholding post-change estimation
AbstractAbstractThe online parallel monitoring problem is studied when one is monitoring large-scale data streams, and an event occurs at an unknown time and affects an unknown subset of data streams. Efficient online parallel monitoring schemes are developed by combining the standard sequential change-point method with hard-thresholding post-change estimation. Theoretical analysis and simulation study demonstrate the usefulness of hard-thresholding for online parallel monitoring.Discussion on "Change-Points : From Sequential Detection to Biology and Back" by David O. Siegmund
AbstractMei, Y. (n.d.).Publication year
2013Journal title
Sequential AnalysisVolume
32Issue
1Page(s)
32-35AbstractIn his interesting paper, Professor Siegmund illustrates that the problem formulations and methodologies are generally transferable between off-line and on-line settings of change-point problems. In our discussion of his paper, we echo his thoughts with our own experiences.Quantization effect on the log-likelihood ratio and its application to decentralized sequential detection
AbstractWang, Y., & Mei, Y. (n.d.).Publication year
2013Journal title
IEEE Transactions on Signal ProcessingVolume
61Issue
6Page(s)
1536-1543AbstractIt is well known that quantization cannot increase the Kullback-Leibler divergence which can be thought of as the expected value or first moment of the log-likelihood ratio. In this paper, we investigate the quantization effects on the second moment of the log-likelihood ratio. It is shown via the convex domination technique that quantization may result in an increase in the case of the second moment, but the increase is bounded above by 2/e. The result is then applied to decentralized sequential detection problems not only to provide simpler sufficient conditions for asymptotic optimality theories in the simplest models, but also to shed new light on more complicated models. In addition, some brief remarks on other higher-order moments of the log-likelihood ratio are also provided.A multistage procedure for decentralized sequential multi-hypothesis testing problems
AbstractWang, Y., & Mei, Y. (n.d.).Publication year
2012Journal title
Sequential AnalysisVolume
31Issue
4Page(s)
505-527AbstractWe studied the problem of sequentially testing M ≥ 2 hypotheses with a decentralized sensor network system. In such a system, the local sensors observe raw data and then send quantized observations to a fusion center, which makes a final decision regarding hypothesis is true. Motivated by the two-stage tests in Wang and Mei (2011), we propose a multistage decentralized sequential test that provides multiple opportunities for the local sensors to adjust to the optimal local quantizers. It is demonstrated that when the hypothesis testing problem is asymmetric, the multistage test is second-order asymptotically optimal. Even though this result constitutes an interesting theoretical improvement over twostage tests that can enjoy only first-order asymptotic optimality, the corresponding practical merits seem to be only marginal. Indeed, performance gains over two-stage procedures with carefully selected thresholds are small.Quantization effect on second moment of log-likelihood ratio and its application to decentralized sequential detection
AbstractAbstractIt is well known that quantization cannot increase the Kullback-Leibler divergence which can be thought of as the expected value or first moment of the log-likelihood ratio. In this paper, we investigate the quantization effects on the second moment of the log-likelihood ratio. It is shown that quantization may result in an increase in the case of the second moment, but the increase is bounded above by 2/e. The result is then applied to decentralized sequential detection problems to provide a simpler sufficient condition for asymptotic optimality theory, and the technique is also extended to investigate the quantization effects on other higher-order moments of the log-likelihood ratio and provide lower bounds on higher-order moments.Asymptotic optimality theory for decentralized sequential multihypothesis testing problems
AbstractWang, Y., & Mei, Y. (n.d.).Publication year
2011Journal title
IEEE Transactions on Information TheoryVolume
57Issue
10Page(s)
7068-7083AbstractThe Bayesian formulation of sequentially testing M ≥ 3 hypotheses is studied in the context of a decentralized sensor network system. In such a system, local sensors observe raw observations and send quantized sensor messages to a fusion center which makes a final decision when stopping taking observations. Asymptotically optimal decentralized sequential tests are developed from a class of "two-stage" tests that allows the sensor network system to make a preliminary decision in the first stage and then optimize each local sensor quantizer accordingly in the second stage. It is shown that the optimal local quantizer at each local sensor in the second stage can be defined as a maximin quantizer which turns out to be a randomization of at most M-1 unambiguous likelihood quantizers (ULQ). We first present in detail our results for the system with a single sensor and binary sensor messages, and then extend to more general cases involving any finite alphabet sensor messages, multiple sensors, or composite hypotheses.Early detection of a change in poisson rate after accounting for population size effects
AbstractMei, Y., Won Han, S., & Tsui, K. L. (n.d.).Publication year
2011Journal title
Statistica SinicaVolume
21Issue
2Page(s)
597-624AbstractMotivated by applications in bio and syndromic surveillance, this article is concerned with the problem of detecting a change in the mean of Poisson distributions after taking into account the effects of population size. The family of generalized likelihood ratio (GLR) schemes is proposed and its asymptotic optimality properties are established under the classical asymptotic setting. However, numerical simulation studies illustrate that the GLR schemes are at times not as efficient as two families of ad-hoc schemes based on either the weighted likelihood ratios or the adaptive threshold method that adjust the effects of population sizes. To explain this, a further asymptotic optimality analysis is developed under a new asymptotic setting that is more suitable to our finite-sample numerical simulations. In addition, we extend our approaches to a general setting with arbitrary probability distributions, as well as to the continuous-time setting involving the multiplicative intensity models for Poisson processes, but further research is needed.Quickest detection in censoring sensor networks
AbstractAbstractThe quickest change detection problem is studied in a general context of monitoring a large number of data streams in sensor networks when the trigger event may affect different sensors differently. In particular, the occurring event could have an immediate or delayed impact on some unknown, but not necessarily all, sensors. Motivated by censoring sensor networks, scalable detection schemes are developed based on the sum of those local CUSUM statistics that are large under either hard thresholding or top-r thresholding rules or both. The proposed schemes are shown to possess certain asymptotic optimality properties.Decentralized multihypothesis sequential detection
AbstractAbstractThis article is concerned with decentralized sequential testing of multiple hypotheses. In a sensor network system with limited local memory, raw observations are observed at the local sensors, and quantized into binary sensor messages that are sent to a fusion center, which makes a final decision. It is assumed that the raw sensor observations are distributed according to a set of M ≥ 2 specified distributions, and the fusion center has to utilize quantized sensor messages to decide which one is the true distribution. Asymptotically Bayes tests are offered for decentralized multihypothesis sequential detection by combining three existing methodologies together: tandem quantizers, unambiguous likelihood quantizers, and randomized quantizers.Discussion on "Quickest detection problems : Fifty years later" by Albert N. Shiryaev
AbstractMei, Y. (n.d.).Publication year
2010Journal title
Sequential AnalysisVolume
29Issue
4Page(s)
410-414AbstractIn his interesting article Professor Shiryaev reviewed how he got motivated from real-applications of target detection in radar systems to develop sequential changepoint detection theory and also described different approaches to formulate mathematical problems. In our discussion of his article we focus on a couple of new real-world applications of sequential change-point detection and some new challenges as well.Efficient scalable schemes for monitoring a large number of data streams
AbstractMei, Y., & Mei, Y. (n.d.).Publication year
2010Journal title
BiometrikaVolume
97Issue
2Page(s)
419-433AbstractThe sequential changepoint detection problem is studied in the context of global online monitoring of a large number of independent data streams. We are interested in detecting an occurring event as soon as possible, but we do not know when the event will occur, nor do we know which subset of data streams will be affected by the event. A family of scalable schemes is proposed based on the sum of the local cumulative sum, cusum, statistics from each individual data stream, and is shown to asymptotically minimize the detection delays for each and every possible combination of affected data streams, subject to the global false alarm constraint. The usefulness and limitations of our asymptotic optimality results are illustrated by numerical simulations and heuristic arguments. The Appendices contain a probabilistic result on the first epoch to simultaneous record values for multiple independent random walks.Decentralized two-sided sequential tests for a normal mean
AbstractAbstractThis article is concerned with decentralized sequential testing of a normal mean μ with two-sided alternatives. It is assumed that in a single-sensor network system with limited local memory, i.i.d, normal raw observations are observed at the local sensor, and quantized into binary messages that are sent to the fusion center, which makes a final decision between the null hypothesis Ho : μ = 0 and the alternative hypothesis HI : μ == ±1. We propose a decentralized sequential test using the idea of tandem quantizers (or equivalently, a oneshot feedback). Surprisingly, our proposed test only uses the quantizers of the form I(Xn ≥ λ), but it is shown to be asymptotically Bayes. Moreover, by adopting the principle of invariance, we also investigate decentralized invariant tests with the stationary quantizers of the form I (|Xn| ≤ λ), and show that λ = 0.5 only leads to a suboptimal decentralized invariant sequential test. Numerical simulations are conducted to support our arguments.Linear-mixed effects models for feature selection in high-dimensional NMR spectra
AbstractMei, Y., Kim, S. B., & Tsui, K. L. (n.d.).Publication year
2009Journal title
Expert Systems with ApplicationsVolume
36Issue
3 PART 1Page(s)
4703-4708AbstractFeature selection in metabolomics can identify important metabolite features that play a significant role in discriminating between various conditions among samples. In this paper, we propose an efficient feature selection method for high-resolution nuclear magnetic resonance (NMR) spectra obtained from time-course experiments. Our proposed approach combines linear-mixed effects (LME) models with a multiple testing procedure based on a false discovery rate. The proposed LME approach is illustrated using NMR spectra with 574 metabolite features obtained for an experiment to examine metabolic changes in response to sulfur amino acid intake. The experimental results showed that classification models constructed with the features selected by the proposed approach resulted in lower rates of misclassification than those models with full features. Furthermore, we compared the LME approach with the two-sample t-test approach that oversimplifies the time-course factor.A comparison of methods for determining HIV viral set point
AbstractMei, Y., Wang, L., Holte, S. E., & Mei, Y. (n.d.).Publication year
2008Journal title
Statistics in MedicineVolume
27Issue
1Page(s)
121-139AbstractDuring a course of human immunodeficiency virus (HIV-1) infection, the viral load usually increases sharply to a peak following infection and then drops rapidly to a steady state, where it remains until progression to AIDS. This steady state is often referred to as the viral set point. It is believed that the HIV viral set point results from an equilibrium between the HIV virus and immune response and is an important indicator of AIDS disease progression. In this paper, we analyze a real data set of viral loads measured before antiretroviral therapy is initiated, and propose two-phase regression models to utilize all available data to estimate the viral set point. The advantages of the proposed methods are illustrated by comparing them with two empirical methods, and the reason behind the improvement is also studied. Our results illustrate that for our data set, the viral load data are highly correlated and it is cost effective to estimate the viral set point based on one or two measurements obtained between 5 and 12 months after HIV infection. The utility and limitations of this recommendation will be discussed.Asymptotic optimality theory for decentralized sequential hypothesis testing in sensor networks
AbstractMei, Y. (n.d.).Publication year
2008Journal title
IEEE Transactions on Information TheoryVolume
54Issue
5Page(s)
2072-2089AbstractThe decentralized sequential hypothesis testing problem is studied in sensor networks, where a set of sensors receive independent observations and send summary messages to the fusion center, which makes a final decision. In the scenario where the sensors have full access to their past observations, the first asymptotically Bayes sequential test is developed having the same asymptotic performance as the optimal centralized test that has access to all sensor observations. Next, in the scenario where the sensors do not have full access to their past observations, a simple but asymptotically Bayes sequential tests is developed, in which sensor message functions are what we call tandem quantizer, where each sensor only uses two different sensor quantizers with at most one switch between these two possibilities. Moreover, a new minimax formulation of optimal stationary sensor quantizers is proposed and is studied in detail in the case of additive Gaussian sensor noise. Finally, our results show that in the simplest models, feedback from the fusion center does not improve asymptotic performance in the scenario with full local memory, however, even a one-shot, one-bit feedback can significantly improve performance in the case of limited local memory.Author's responses
AbstractMei, Y. (n.d.).Publication year
2008Journal title
Sequential AnalysisVolume
27Issue
4Page(s)
414-419AbstractIn this rejoinder I briefly summarize my thoughts on appropriate measures of performance for evaluating change-point detection schemes, particularly the false alarm criterion. Then I address some specific issues in the light of the discussion pieces from eight experts in this field.Is average run length to false alarm always an informative criterion?
AbstractMei, Y. (n.d.).Publication year
2008Journal title
Sequential AnalysisVolume
27Issue
4Page(s)
354-376AbstractApart from Bayesian approaches, the average run length (ARL) to false alarm has always been seen as the natural performance criterion for quantifying the propensity of a detection scheme to make false alarms, and no researchers seem to have questioned this on grounds that it does not always apply. In this article, we show that in the change-point problem with mixture prechange models, detection schemes with finite detection delays can have infinite ARLs to false alarm. We also discuss the implication of our results on the change-point problem with either exchangeable prechange models or hidden Markov models. Alternative minimax formulations with different false alarm criteria are proposed.Optimal stationary binary quantizer for decentralized quickest change detection in hidden Markov models
AbstractAbstractThe decentralized quickest change detection problem is studied in sensor networks, where a set of sensors receive observations from a hidden Markov model X and send sensor messages to a central processor, called the fusion center, which makes a final decision when observations are stopped. It is assumed that the parameter θ in the hidden Markov model for X changes from θ0 to θ1 at some unknown time. The problem is to determine the policies at the sensor and fusion center levels to jointly optimize the detection delay subject to the average run length (ARL) to false alarm constraint. In this article, a CUSUM-type fusion rule with stationary binary sensor messages is studied and a simple method for choosing the optimal local sensor thresholds is introduced. Further research is also given.Sample size calculation for the van Elteren test adjusting for ties
AbstractZhao, Y. D., Rahardja, D., & Mei, Y. (n.d.).Publication year
2008Journal title
Journal of Biopharmaceutical StatisticsVolume
18Issue
6Page(s)
1112-1119AbstractIn this article we study sample size calculation methods for the asymptotic van Elteren test. Because the existing methods are only applicable to continuous data without ties, in this article we develop a new method that can be used on ordinal data. The new method has a closed form formula and is very easy to calculate. The new sample size formula performs very well because our simulations show that the corresponding actual powers are close to the nominal powers.A discussion on "Detection of intrusions in information systems by sequential change-point methods" by Tartakovsky, Rozovskii, Blažek, and Kim
AbstractMei, Y. (n.d.).Publication year
2006Journal title
Statistical MethodologyVolume
3Issue
3Page(s)
304-306Abstract~Comments on "a note on optimal detection of a change in distribution," by benjamin Yakir
AbstractMei, Y. (n.d.).Publication year
2006Journal title
Annals of StatisticsVolume
34Issue
3Page(s)
1570-1576AbstractThe purpose of this note is to show that in a widely cited paper by Yakir [Ann. Statist. 25 (1997) 2117-2126], the proof that the so-called modified Shiryayev-Roberts procedure is exactly optimal is incorrect. We also clarify the issues involved by both mathematical arguments and a simulation study. The correctness of the theorem remains in doubt.Information bounds for decentralized sequential detection
AbstractAbstractThe main purpose of this paper is to develop an asymptotic theory for the decentralized sequential hypothesis testing problems under the frequentist framework. Sharp asymptotic bounds on the average sample numbers or sample sizes of sequential or fixed-sample tests are provided in the decentralized decision systems in different scenarios subject to error probabilities constraints. Asymptotically optimal tests are offered in the system with full local memory. Optimal binary quantizers are also studied in the case of additive Gaussian sensor noises.