Channel Title: Generable Channel Description: The latest advances in Bayesian computing allow users to estimate models that were previously out of reach for most practitioners in industry and academia. Today, we are able to fit models with full Bayesian inference that jointly estimate hundreds of thousands and sometimes millions of parameters. As model complexity grows, we need tools to make sense of these models so we can better understand their strengths and more importantly their weaknesses. By analogy, if we are building a plane, it is our responsibility to test under which conditions it can and cannot fly. We owe this much to our p̶a̶s̶s̶e̶n̶g̶e̶r̶s̶ users. In this channel, we are focusing on understanding and explaining uncertainty in the broadest sense of the word including interesting model structures, model inferences, predictions, causal inference, decision analysis, and communicating models and uncertainty. <video> Paul Bürkner: An introduction to Bayesian multilevel modeling with brms Description: The talk is about Bayesian multilevel models and their implementation in R using the package brms. It starts with a short introduction to multilevel modeling and to Bayesian statistics in general followed by an introduction to Stan, which is a flexible language for fitting open-ended Bayesian models. We then explain how to access Stan using the standard R formula syntax via the brms package. The package supports a wide range of response distributions and modeling options such as splines, autocorrelation, and censoring all in a multilevel context. A lot of post-processing and plotting methods are implemented as well. Some examples from Psychology and Medicine will be discussed. Paul Bürkner is a statistician currently working as a Junior Research Group Leader at the Cluster of Excellence SimTech at the University of Stuttgart (Germany). He is the author of the R package brms and a member of the Stan Development Team. Previously, he studied Psychology and Mathematics at the Universities of Münster and Hagen (Germany) and did his PhD in Münster on optimal design and Bayesian data analysis. He has also worked as a Postdoctoral researcher at the Department of Computer Science at Aalto University (Finland). Pual's website: https://paul-buerkner.github.io/about/ brms: https://github.com/paul-buerkner/brms </video> <video> Jessica Hullman: Theories of Inference for Data Interactions Description: Research and development in computer science and statistics have produced increasingly sophisticated software interfaces for interactive and exploratory analysis, optimized for easy pattern finding and data exposure. But design philosophies that emphasize exploration over other phases of analysis risk confusing a need for flexibility with a conclusion that exploratory visual analysis is inherently “model-free” and cannot be formalized. I will motivate how without a grounding in theories of human statistical inference, research in exploratory visual analysis can lead to contradictory interface objectives and representations of uncertainty that can discourage users from drawing valid inferences. I will discuss how the concept of a model check in a Bayesian statistical framework unites exploratory and confirmatory analysis, and how this understanding relates to other proposed theories of graphical inference. Viewing interactive analysis as driven by model checks suggests new directions for software and empirical research around exploratory and visual analysis, as well as important questions about what class of problems visual analysis is suited to answer. Jessica Hullman is an Associate Professor of Computer Science at Northwestern University. Her research looks at how to design, evaluate, coordinate, and theorize representations for data-driven decision-making. She co-directs the Midwest Uncertainty Collective, an interdisciplinary group of researchers working on topics in visualization, uncertainty communication, and human-in-the-loop data analysis, with Matt Kay. Jessica is the recipient of a Microsoft Faculty Fellowship, NSF CAREER Award, and multiple best papers at top visualization and human-computer interaction conferences. Jessica Hullman: http://users.eecs.northwestern.edu/~jhullman/ Designing for interactive exploratory data analysis requires theories of graphical inference: http://www.stat.columbia.edu/~gelman/research/unpublished/EDA_theories_of_inference.pdf </video> <video> Tamara Broderick: Fast Discovery of Pairwise Interactions in High Dimensions using Bayes Description: Discovering interaction effects on a response of interest is a fundamental problem in medicine, economics, and many other disciplines. In theory, Bayesian methods for discovering pairwise interactions enjoy many benefits such as coherent uncertainty quantification, the ability to incorporate background knowledge, and desirable shrinkage properties. In practice, however, Bayesian methods are often computationally intractable for problems of even moderate dimension p. Our key insight is that many hierarchical models of practical interest admit a particular Gaussian process (GP) representation; the GP allows us to capture the posterior with a vector of O(p) kernel hyper-parameters rather than O(p^2) interactions and main effects. With the implicit representation, we can run Markov chain Monte Carlo (MCMC) over model hyper-parameters in time and memory linear in p per iteration. We focus on sparsity-inducing models; on datasets with a variety of covariate behaviors, we show that our method: (1) reduces runtime by orders of magnitude over naive applications of MCMC, (2) provides lower Type I and Type II error relative to state-of-the-art LASSO-based approaches, and (3) offers improved computational scaling in high dimensions relative to existing Bayesian and LASSO-based approaches. Tamara Broderick is an Associate Professor in the Department of Electrical Engineering and Computer Science at MIT. She is a member of the MIT Computer Science and Artificial Intelligence Laboratory (CSAIL), the MIT Statistics and Data Science Center, and the Institute for Data, Systems, and Society (IDSS). She completed her Ph.D. in Statistics at the University of California, Berkeley in 2014. Previously, she received an AB in Mathematics from Princeton University (2007), a Master of Advanced Study for completion of Part III of the Mathematical Tripos from the University of Cambridge (2008), an MPhil by research in Physics from the University of Cambridge (2009), and an MS in Computer Science from the University of California, Berkeley (2013). Her recent research has focused on developing and analyzing models for scalable Bayesian machine learning. She has been awarded selection to the COPSS Leadership Academy (2021), an Early Career Grant (ECG) from the Office of Naval Research (2020), an AISTATS Notable Paper Award (2019), an NSF CAREER Award (2018), a Sloan Research Fellowship (2018), an Army Research Office Young Investigator Program (YIP) award (2017), Google Faculty Research Awards, an Amazon Research Award, the ISBA Lifetime Members Junior Researcher Award, the Savage Award (for an outstanding doctoral dissertation in Bayesian theory and methods), the Evelyn Fix Memorial Medal and Citation (for the Ph.D. student on the Berkeley campus showing the greatest promise in statistical research), the Berkeley Fellowship, an NSF Graduate Research Fellowship, a Marshall Scholarship, and the Phi Beta Kappa Prize (for the graduating Princeton senior with the highest academic average). </video> <video> Juho Timonen: Design of Statistical Modeling Software Description: Abstract Juho presents what he thinks is an ideal modular design for statistical modeling software, based on his experiences as a user, developer, and researcher. He looks at the structure of existing software, focusing on high-level user interfaces that use Stan as their inference engine. Juho uses the longitudinal Gaussian process modeling package lgpr and its future improvements to demonstrate different concepts and challenges related to software design and development. About the speaker Juho Timonen completed his Bachelor’s degree majoring in Mathematics and Systems Analysis at Aalto University, Finland. His first research project was in game theory, and in his Bachelor’s thesis, Juho studied mixed-strategy equilibria in repeated games. He went on to do his Master’s degree at Aalto, majoring in Applied Mathematics. Juho joined the Computational Systems Biology group in the CS department at Aalto, where he got involved in probabilistic modeling and did his Master’s thesis on ODE modeling of gene regulatory networks. Juho continued as a Doctoral student in the same group, where his research is focused on probabilistic modeling. </video> <video> Elea Feit: A Gaussian Process Model for Response Time in Conjoint Surveys Description: Choice-based conjoint analysis is a widely-used technique for assessing consumer preferences. By observing how customers choose between alternatives with varying attributes, consumers' preferences for the attributes can be inferred. When one alternative is chosen over the others, we know that the decision-maker perceived this option to have higher utility compared to the unchosen options. In addition to observing the choice that a customer makes, we can also observe the response time for each task. Building on extant literature, we propose a Gaussian Process model that relates response time to four features of the choice task (question number, alternative difference, alternative attractiveness, and attribute difference). We discuss the nonlinear relationships between these four features and response time and show that incorporating response time into the choice model provides us with a better understanding of individual preferences and improves our ability to predict choices. Elea Feit is an Associate Professor of Marketing at Drexel University. Prior to joining Drexel, she spent most of her career at the boundary between academia and industry, including positions at General Motors Research, The Modellers, and Wharton Customer Analytics. Her work is inspired by the decision problems that marketers face and she has published research on using randomized experiments to measure advertising incrementality and using conjoint analysis to design new products. Methodologically, she is a Bayesian with expertise in hierarchical models, experimental design, missing data, data fusion, and decision theory. She is also the co-author of R for Marketing Research and Analytics. More at eleafeit.com. </video> <video> Rok Češnovar: The Current State and Evolution of Stan Description: We will present the current state of the Stan ecosystem, highlight some of the advances that improved the performance of Stan in the past few years, and discuss what is still to come in the not-so-distant future. You will hear about the core modules of Stan, how they all fit together, and how the various interfaces bring that core to life in different ways to make your Stan models run. If your models have been compiling and running faster recently and you were wondering why, we will present the improvements that were made in Stan over the last few years that sped up those pesky gradient evaluations. If you feel that your models are still running slowly, we will provide a few tips on how to identify the computational (non-modeling) reasons why that may be the case, and what are some of the things you can do to speed up computation. Rok Češnovar is a Stan developer and a PhD student at the University of Ljubljana who is interested in making Bayesian inference faster using parallelism on GPUs and multi-core CPUs. He joined the Stan development team while working on adding GPU support with the team of prof. Štrumbelj at UL, but quickly became interested in almost all aspects of Stan. In addition to working on adding the GPU support, he has co-authored the cmdstanr R package, added profiling for Stan, and implemented the language side of reduce_sum and the new ODE interface. He has also been managing and coordinating Stan & CmdStan releases for the last two years. </video> <video> Lizzie Wolkovivh: Predicting future forest tree communities and winegrowing regions with Stan Description: Climate change is having large impacts on natural and agricultural systems around the globe. Mitigating the worst consequences requires models that mechanistically predict changes. Towards that goal, the Temporal Ecology Lab works on models to better predict the most reported biological impact – shifts in phenology, the timing of recurring life history events such as leafout, and flowering. Here I review three major areas of research where Bayesian inference has been critical to my lab’s insights and advances: declining plant sensitivity to warming temperatures over time and space, mismatches between critical species interactions (for example, plants and pollinators), and shifting winegrowing regions with warming. Elizabeth Wolkovich is an Associate Professor and Canada Research Chair at the University of British Columbia where she runs the Temporal Ecology Lab. Her research focuses on understanding how climate change shapes plants and plant communities, with a focus on shifts in the timing of seasonal development. She is particularly interested in how climate change will affect different winegrape varieties, and how shifting varieties may help growers adapt to warming. Elizabeth Wolkovich: https://biodiversity.ubc.ca/people/faculty/elizabeth-m-wolkovich Temporal Ecology Lab: https://temporalecology.org/people/ </video> <video> Charles Margossian: Some Outstanding Challenges when Solving ODEs in a Bayesian context Description: Many scientific models rely on differential equation-based likelihoods. Some unique challenges arise when fitting such models with full Bayesian inference. Indeed as our algorithm (e.g. Markov chain Monte Carlo) explores the parameter space, we must solve, not one, but a range of ODEs whose behaviors can change dramatically with different parameter values. We will present two examples where this phenomenon occurs: a classical mechanics problem where the speed of the solver differs for different parameter values; and a pharmacology example, wherein the ODE behaves in a stiff manner during the warmup phase but becomes non-stiff during the sampling phase. We'll then have a candid discussion about the difficulties that arise when developing a modeling workflow with ODE-based models and brainstorm some ideas on how to move forward. Charles Margossian is a PhD student in statistics at Columbia Univeristy, advised by Andrew Gelman. He is interested in the development and application of novel statistical methods to the life and physical sciences. Much of his work focuses on Bayesian modeling and its computational implementation. Charles is a developer of the probabilistic programing language Stan and the co-creator of Torsten, an extension of Stan for biomedical applications. </video> <video> Duco Veen: On using expert information in Bayesian statistics Description: Duco Veen is an Assistant Professor at the Department of Global Health situated at the Julius Center for Health Sciences and Primary Care of the University Medical Center Utrecht. In that capacity, he is involved in the COVID-RED, AI for Health, and Trials@Home projects. In addition, he is appointed as Extraordinary Professor at the Optentia Research Programme of North-West University, South Africa. Duco works on the development of ShinyStan and has been elected as a member of the Stan Governing Body. In this talk, Duco will discuss how expert knowledge can be captured and formulated as prior information for Bayesian analyses. This process is also called expert elicitation. He will highlight some ways to improve the quality of the expert elicitation and provide case examples of work he did with his colleagues. Additionally, Duco will discuss how the captured expert knowledge can be contrasted with evidence provided by traditional data collection methods or other prior information. This can be done, for instance, for reasons of quality control or training purposes where experts reflect on their provided information. </video> <video> Jonathan Auerbach: Could voting restrictions be increasing election fraud? Description: In this talk, Jonathan will present some research he conducted as the 2020-2021 Science Policy Fellow at the American Statistical Association. The talk will review several projects before focusing on the title question, which Science Policy Director Steve Pierson and Jonathan write about in a recent publication for Statistics and Public Policy. The abstract is summarized below. Jonathan will conclude by challenging viewers to enter an exciting new prediction competition that will be run by the George Mason University Statistics Department in February. In the paper, we estimate the change in the reported number of voter fraud cases when states switch to conducting elections by mail. We consider states where many voters have historically received ballots by mail and a subset where registered voters are automatically sent ballots by mail. We compare the number of voter fraud cases in these states to the number of cases in remaining states using difference in differences and matrix completion (via nuclear-norm penalization). We find no evidence that voting by mail increases the risk of voter fraud overall. In fact, we estimate Washington would have reported 73 more cases of fraud between 2011 and 2019 had it not introduced its vote-by-mail law. About the speaker ----------------------------- Jonathan Auerbach is an Assistant Professor in the Department of Statistics at George Mason University. His research covers a wide range of topics at the intersection of statistics and public policy. Previously, Auerbach served as a science policy fellow at the American Statistical Association. He was also a researcher at the Center for Urban Research at the City University of New York and an analyst for New York City’s legislature, the City Council. </video> <video> Uri Shalit: Towards responsible patient-level causal inference: taking uncertainty seriously Description: Topics: Calibrated Webcast, Bayesian Inference, Causal Inference, Machine Learning, Healthcare About the speaker ----------------------------- Uri Shalit is an Assistant Professor in the Faculty of Industrial Engineering and Management at Technion University. He received his Ph.D. in Machine Learning and Neural Computation from the Hebrew University in 2015. Prior to joining Technion, Uri was a postdoctoral researcher at NYU working with prof. David Sontag. Uri's research is currently focused on three subjects. The first is applying machine learning to the field of healthcare, especially in terms of providing physicians with decision support tools based on big health data. The second subject Uri is interested in is the intersection of machine learning and causal inference, especially the problem of learning individual-level effects. Finally, Uri is working on bringing ideas from causal inference into the field of machine learning, focusing on problems in robust learning, transfer learning, and interoperability. Abstract -------------- A plethora of new methods for estimating patient-level causal effects have been proposed recently, focusing on what is technically known as (high-dimensional) conditional average effects (CATE). The intended use of many of these methods is to inform human decision-makers about the probable outcomes of possible actions, for example, clinicians choosing among different medications for a patient. For such high-stakes decisions, it is crucial for any algorithm to responsibly convey a measure of uncertainty about its output, in order to enable informed decision making on the side of the human and to avoid catastrophic errors. We will discuss recent work where we present new methods for conveying uncertainty in CATE estimation stemming from several distinct sources: (i) finite data (ii) covariate shift (iii) violations of the overlap assumption (iv) violation of the no-hidden confounders assumption. We show how these measures of uncertainty can be used to responsibly decide when to defer decisions to experts and avoid unwarranted errors. This is joint work with Andrew Jesson, Sören Mindermann, and Yarin Gal of Oxford University. </video> <video> Aki Vehtari: On Bayesian Workflow Description: We discuss some parts of the Bayesian workflow with a focus on the need and justification for an iterative process. The talk is partly based on a review paper by Gelman, Vehtari, Simpson, Margossian, Carpenter, Yao, Kennedy, Gabry, Bürkner, and Modrák with the following abstract: "The Bayesian approach to data analysis provides a powerful way to handle uncertainty in all observations, model parameters, and model structure using probability theory. Probabilistic programming languages make it easier to specify and fit Bayesian models, but this still leaves us with many options regarding constructing, evaluating, and using these models, along with many remaining challenges in computation. Using Bayesian inference to solve real-world problems requires not only statistical skills, subject matter knowledge, and programming, but also awareness of the decisions made in the process of data analysis. All of these aspects can be understood as part of a tangled workflow of applied Bayesian statistics. Beyond inference, the workflow also includes iterative model building, model checking, validation and troubleshooting of computational problems, model understanding, and model comparison. We review all these aspects of workflow in the context of several examples, keeping in mind that applied research can involve fitting many models for any given problem, even if only a subset of them are relevant once the analysis is over." </video> <video> Arman Oganisian: Introduction to Nonparametric Bayes Description: Bayesian nonparametrics combines the flexibility often associated with machine learning with principled uncertainty quantification required for inference. Popular priors in this class include Gaussian Processes, Bayesian Additive Regression Trees, Chinese Restaurant Processes, and more. But what exactly are “nonparametric” priors? How can we compute posteriors under such priors? And how can we use them for flexible modeling? This talk will explore these questions by introducing nonparametric Bayes at a conceptual level and walking through a few common priors, with a particular focus on the Dirichlet Process prior for regression. Sorry about the video problems at the beginning. We are working to sort that out in future videos. Arman is an Assistant Professor of Biostaistics (tenure-track) at Brown University’s Department of Biostatistics. His methodological research centers around developing nonparametric Bayesian methods for flexibly estimating causal effects with observational data. This involves, among other things, devising priors over high-dimensional model spaces and constructing MCMC methods for efficient posterior computation. Though broadly applicable, these methodological interests are motivated by applied problems in health economics and, more recently, statistical challenges in oncology research. Arman's website: https://stablemarkets.netlify.app/ </video> <video> Kristian Brock: Functional uniform priors for dose-response models Description: Dose-response modeling frequently employs non-linear regression. Functional uniform priors are distributions that can be derived for parameters that convey approximate uniformity over the range of function shapes generated by the model. They provide a stark alternative to regular uniform priors, which in the non-linear setting can provide potentially undue influence on the estimated functional form. Using methods introduced by Bornkamp, we provide full analytical derivations of functional uniform priors for a range of non-linear dose-response models. We then examine the numerical performance of these two types of prior, and analogous maximum likelihood models, in a simulation study. We also investigate the incidence of several markers that question the adequacy of model fit, in simulated and real phase I clinical trial datasets. We show that mean absolute errors of response estimates are smaller when using functional uniform priors instead of regular uniform priors. This effect was seen in all simulation scenarios at all sample sizes. Irrespective of the decreases in errors, biases tended to be slightly larger under functional uniform priors. When the analysis model and data generating model matched, functional uniform priors yielded very close to nominal coverage probabilities, even at the smallest sample size. This was not true for models that used regular uniform priors. Markers of model fit inadequacy using real and simulated datasets were much more common in maximum likelihood models, with parameters being estimated near the boundary being a notable problem. Functional uniform priors provide a general improvement over regular uniform priors in Bayesian dose-response modeling and should be preferred. The substantial mathematical work required to derive the priors is abrogated by the results derived in this research. Kristian Brock is a biostatistician specializing in Bayesian and early-phase trial methodology at UCB. He has experience in cancer and non-cancer clinical trials. He has authored several methodology papers and conducted the analysis for several trial results publications. By honourary association, Kristian leads an MSc module on early phase trials at the University of Birmingham and supervises PhDs on trial methodology. </video> <video> Lester Mackey: Kernel Thinning and Stein Thinning Description: Abstract This talk will introduce two new tools for summarizing a probability distribution more effectively than independent sampling or standard Markov chain Monte Carlo thinning: Given an initial n point summary (for example, from independent sampling or a Markov chain), kernel thinning finds a subset of only square-root n points with comparable worst-case integration error across a reproducing kernel Hilbert space. If the initial summary suffers from biases due to off-target sampling, tempering, or burn-in, Stein thinning simultaneously compresses the summary and improves the accuracy by correcting for these biases. These tools are especially well-suited for tasks that incur substantial downstream computation costs per summary point like organ and tissue modeling in which each simulation consumes 1000s of CPU hours. About the speaker Lester Mackey is a Principal Researcher at Microsoft Research, where he develops machine learning methods, models, and theory for large-scale learning tasks driven by applications from healthcare, climate forecasting, and the social good. Lester moved to Microsoft from Stanford University, where he was an assistant professor of Statistics and (by courtesy) of Computer Science. He earned his Ph.D. in Computer Science and MA in Statistics from UC Berkeley and his BSE in Computer Science from Princeton University. He co-organized the second-place team in the Netflix Prize competition for collaborative filtering, won the Prize4Life ALS disease progression prediction challenge, won prizes for temperature and precipitation forecasting in the yearlong real-time Subseasonal Climate Forecast Rodeo, and received best paper and best student paper awards from the ACM Conference on Programming Language Design and Implementation and the International Conference on Machine Learning. </video>

Contacts and Links

Description

The latest advances in Bayesian computing allow users to estimate models that were previously out of reach for most practitioners in industry and academia. Today, we are able to fit models with full Bayesian inference that jointly estimate hundreds of thousands and sometimes millions of parameters. As model complexity grows, we need tools to make sense of these models so we can better understand their strengths and more importantly their weaknesses. By analogy, if we are building a plane, it is our responsibility to test under which conditions it can and cannot fly. We owe this much to our p̶a̶s̶s̶e̶n̶g̶e̶r̶s̶ users.

In this channel, we are focusing on understanding and explaining uncertainty in the broadest sense of the word including interesting model structures, model inferences, predictions, causal inference, decision analysis, and communicating models and uncertainty.

Videos from channel