If significant, clearly report which group mean is higher, along with the effect size.
Before conducting analysis, we need to ensure that we will have an adequate sample size to detect an effect. Sample size relates to the concept of power. For example, to detect a small effect, a larger sample is needed. Larger sample sizes can thus detect a smaller effect. Sample size is determined through a power analysis. The determination of sample size is never a simple percent of the population, but a calculated number based on the planned statistical tests, significance level and effect size. 8 I recommend using G*Power for basic power calculations, although many other options are available. In the exemplar study, the authors did not report their power analysis prior to conducting the study, but they gave a post-hoc power analysis of the actual power based on their sample size and the effect size detected. 6
Data often need cleaning and other preparation before conducting analysis. Problems requiring cleaning include values outside of an acceptable range and missing values. Any particular value could be wrong because of a data entry error or data collection problem. Visually inspecting data can reveal anomalies. For example, an age value of 200 is clearly an error, or a value of 9 on a 1–5 Likert-type scale is an error. An easy way to start inspecting data is to sort each variable by ascending values and then descending values to look for atypical values. Then, try to correct the problem by determining what the value should be. Missing values are a more complicated problem because a concern is why the value is missing. A few missing values at random is not necessarily a concern, but a pattern of missing values (eg, individuals from a specific ethnic group tend to skip a certain question) indicates a systematic missingness that could indicate a problem with the data collection instrument. Descriptive statistics are an additional way to check for errors and ensure data are ready for analysis. While not discussed in the communication assessment exemplar, the authors did prepare data for analysis and report missing values in their descriptive statistics.
Before running inferential statistics, it is critical to first describe the data. Obtaining descriptive statistics is a way to check whether data are ready for further analysis. Descriptive statistics give a general sense of trends and can illuminate errors by reviewing frequencies, minimums and maximums that can indicate values outside of the accepted range. Descriptive statistics are also an important step to check whether we meet assumptions for statistical tests. In a quantitative study, descriptive statistics also inform the first table of the results that reports information about the sample, as seen in table 2 of the exemplar study. 6
All statistical tests rely on foundational assumptions. Although some tests are more robust to violations, checking assumptions indicates whether the test is likely to be valid for a particular data set. Foundational parametric statistics (eg, t tests, ANOVA, correlation, regression) assume independent observations and a normal linear distribution of data. In the exemplar study, the authors noted ‘Data from both groups met normality assumptions, based on the Shapiro–Wilk test’ (p508), and gave the statistics in addition to noting specific assumptions for the independent t tests around equality of variances. 6
Conducting the analysis involves running whatever tests were planned. Statistics may be calculated manually or using software like SPSS, Stata, SAS or R. Statistical software provides an output with key tests statistics, p values that indicate whether a result is likely systematic or random, and indicators of fit. In the exemplar study, the authors noted they used SPSS V.22. 6
The first step involves examining whether the statistical model was significant or a good fit. For t tests, ANOVAs, correlation and regression, first examine an overall test of significance. For a t test, if the t statistic is not statistically significant (eg, p>0.05 or a CI crossing 0), we can conclude no significant difference between groups. The communication assessment exemplar reports significance of the t tests along with measures such as equality of variance.
For an ANOVA, if the F statistic is not statistically significant (eg, p>0.05 or a CI crossing 0), we can conclude no significant difference between groups and stop because there is no point in further examining what groups may be different. If the F statistic is significant in an ANOVA, we can then use contrasts or post-hoc tests to examine what is different. For a correlation test, if the r value is not statistically significant (eg, p>0.05 or a CI crossing 0), we can stop because there is no point in looking at the magnitude or direction of the coefficient. If it is significant, we can proceed to interpret the r. Finally, for a regression, we can examine the F statistic as an omnibus test and its significance. If it is not significant, we can stop. If it is significant, then examine the p value of each independent variable and residuals.
When writing statistical results, always start with descriptive statistics and note whether assumptions for tests were met. When reporting inferential statistical tests, give the statistic itself (eg, a F statistic), the measure of significance (p value or CI), the effect size and a brief written interpretation of the statistical test. The interpretation, for example, could note that an intervention was not significantly different from the control or that it was associated with improvement that was statistically significant. For example, the exemplar study gives the pre–post means along standard error, t statistic, p value and an interpretation that postseminar means were lower, along with a reminder to the reader that lower is better. 6
When writing for a journal, follow the journal’s style. Many styles italicise non-Greek statistics (eg, the p value), but follow the particular instructions given. Remember a p value can never be 0 even though some statistical programs round the p to 0. In that case, most styles prefer to report as p<0.001.
Shadish et al 9 provide nine threats to statistical conclusion validity in drawing inferences about the relationship between two variables; the threats can broadly apply to many statistical analyses. Although it helps to consider and anticipate these threats when designing a research study, some only arise after data collection and analysis. Threats to statistical conclusion validity appear in table 4 . 9 Pertinent threats can be dealt with to the extent possible (eg, if assumptions were not met, select another test) and should be discussed as limitations in the research report. For example, in the exemplar study, the authors noted the sample size as a limitation but reported that a post-hoc power analysis found adequate power. 6
Threats to statistical conclusion validity
Threat | Description |
Low statistical power (see step 3) | The sample size is not adequate to detect an effect. |
Violated assumptions of statistical tests (see step 6) | The data violate assumptions needed for the test, such as normality. |
Fishing and error rates | Repeated tests of the same data (eg, multiple comparisons) increase chances of errors in conclusions. |
Unreliability of measures | Error in measurement or instruments can artificially inflate or decrease apparent relationships among variables. |
Restricted range | Statistics can be biased by limited outcome values (eg, high/low only) or floor or ceiling effects in which participants scores are clustered around high or low values. |
Unreliability of treatment implementation | In experiments, unstandardised or inconsistent implementation affects conclusions about correlation. |
Extraneous variance in an experiment | The setting of a study can introduce error. |
Heterogeneity of units | As participants differ within conditions, standard deviation can increase and introduce error, making it harder to detect effects. |
Inaccurate effect size estimation | Outliers or incorrect effect size calculations (eg, a continuous measure for a dichotomous dependent variable) can skew measures of effect. |
Key resources to learn more about statistics include Field 4 and Salkind 10 for foundational information. For advanced statistics, Hair et al 11 and Tabachnick and Fidell 12 provide detailed information on multivariate statistics. Finally, the University of California Los Angeles Institute for Digital Research and Education (stats.idre.ucla.edu/other/annotatedoutput/) provides annotated output from Stata, SAS, Stata and MPlus for many statistical tests to help researchers read the output and understand what it means.
Researchers in family medicine and community health often conduct statistical analyses to address research questions. Following specific steps ensures a systematic and rigorous analysis. Knowledge of these essential statistical procedures will equip family medicine and community health researchers with interpreting literature, reviewing literature and conducting appropriate statistical analysis of their quantitative data.
Nevertheless, I gently remind you that the steps are interrelated, and statistics is not only a consideration at the end of data collection. When designing a quantitative study, investigators should remember that statistics is based on distributions, meaning statistics works with aggregated numerical data and relies on variance within that data to test statistical hypotheses about group differences, relationships or trends. Statistics provides a broad view, based on these distributions, which brings implications at the early design phase. In designing a quantitative study, the nature of statistics generally suggests a larger number of participants in the research (ie, a larger n) to have adequate power to detect statistical significance and draw valid conclusions. Therefore, it will likely be helpful for researchers to include a biostatistician as early as possible in the research team when designing a study.
Contributors: The sole author, TCG, is responsible for the conceptualisation, writing and preparation of this manuscript.
Funding: This study was funded by the National Institutes of Health (10.13039/100000002) and grant number 1K01LM012739.
Competing interests: None declared.
Patient consent for publication: Not required.
Provenance and peer review: Not commissioned; internally peer reviewed.
Don't miss out
Subscribe to STAT+ today, for the best life sciences journalism in the industry
By Robert Metcalf and Jeffrey Francer July 10, 2024
I nnovations in clinical trial designs and tools have the potential to unlock a new era of research that is more convenient for patients, more reflective of real-world treatment conditions, and more likely to enable participation of a diverse set of individuals. But a recent study reveals how far the U.S. is from realizing this potential: regions of the country with the worst social drivers of health are the least likely to host clinical trials.
The disconnect between need and where clinical trials are conducted is a longstanding one. But it was recently highlighted by University of Michigan researchers through an examination of demographic data for people enrolled in clinical trials for new cancer medicines. The most socially vulnerable counties were far less likely to have any nearby trial, a disparity that has worsened over time.
advertisement
Research sponsors and the Food and Drug Administration can respond to this challenge by continuing to support community-based clinical trials. But the regulatory framework that governs these and other modern approaches has not kept pace with innovations in clinical trials and must be updated to enable expansion of trials into more communities.
Clinical trials are essential for establishing the safety and effectiveness of new medicines. Trial results have a greater impact when participants reflect the demographic diversity of those who could potentially benefit from the treatments being evaluated. The University of Michigan research is one more confirmation that the U.S.’s existing clinical trial infrastructure often fails to meet these goals.
Designing and implementing clinical trials is hard work. Reports over time suggest that up to 85% of clinical trials don’t meet their recruitment goals and up to 80% are delayed due to recruitment challenges.
Large-scale clinical trials are typically hosted by large research hospitals and academic institutions, most of which are in big cities. This can exclude people in rural communities from participating in clinical trials, and can present logistical challenges even for individuals who live relatively close to these centers because they may not have the economic means or schedule flexibility to attend multiple appointments.
Today’s clinical trial regulations were created for a different era, when the technology of the time necessitated that studies be conducted at a single location under the direct supervision of an investigator and staff who carried out all aspects of the trial. Participants were required to come to that location. Clinical research still largely relies on this outmoded approach, which frequently requires participants to organize their lives around the trial, and often includes traveling, finding a place to stay, and taking time off from work.
New tools and approaches developed by clinical trial sponsors, working with the FDA, should help make trials more representative. The FDA has signaled an openness to supporting trial designs that make them more accessible for participants, more reflective of real-world conditions, and enable more diverse participation. This modernization of the regulatory framework is critically needed and will contribute to healthier communities by speeding the development of new and better treatments that address unmet medical needs.
Community-based trials, also known as decentralized trials, have the potential to significantly increase participation and diversity in clinical research. By forcing a shift to this model, the Covid-19 pandemic showed just how successful these types of studies can be. To help ensure studies could continue during the pandemic, investigators, trial sponsors, and regulators worked collaboratively during the nationwide shutdown to reverse the process, bringing trials to participants rather than participants to trials.
Lilly, the company we work for, partnered with a leading decentralized research organization to bring our Covid-19 research to at-risk patients in long-term care facilities. An innovative cloud-based system helped recruit participants across multiple sites and make adjustments as needed in real time.
This model allowed Lilly to move quickly, reach more people who were traditionally underrepresented in clinical trials, and protect the health of participants and trial staff during the pandemic, all while maintaining the highest standards of scientific research, patient safety, and data integrity. To be sure, Lilly wasn’t alone in doing this: companies across the biopharmaceutical industry can share similar stories of leveraging innovative, community-based approaches to keep clinical trials running during the Covid-19 crisis.
These updated approaches shouldn’t fade away with the pandemic. Drug developers, investigators, and regulators must build on what was learned. Several key updates to the U.S. clinical trial regulatory framework will be crucial to supporting this progress:
Ease the burden on clinical trial investigators. Enabling better support from sponsor staff can create efficiencies and fill resource needs for community-based providers. Local health care professionals are essential to the success of community-based trials, but most of them do not have the resources or infrastructure to manage many of the demands of clinical studies, such as recruiting participants, providing them with logistical support, and shipping investigational products to them. Trial sponsor staff have the capability to perform tasks like these that involve limited or no contact with participants to avoid conflict of interest. Current regulatory rules, however, provide little guidance on what types of sponsor roles are appropriate, which creates uncertainty for sponsors that can discourage such support.
Update the role of investigators. The shift in clinical trial services to multiple care settings, such as community clinics, mobile medical units, and participants’ homes, must be accompanied by updating how clinical trial investigators provide oversight of these settings. Current regulations state that an investigator must personally conduct or supervise a trial. This requirement can create confusion for a community-based study that includes multiple care settings in numerous communities.
To better accommodate community-based trials without compromising patient safety or data integrity, FDA regulations should be updated to clarify that trial investigators may provide oversight by ensuring that study staff such as local health care providers are appropriately qualified and trained for the trial-related activities they will perform. Such assurance could include confirming proper education and qualifications and meeting state licensing requirements.
Current regulations also state that investigators may administer an investigational product only to study participants they personally supervise. Such regulations do not lend themselves to the flexibility needed to enable community-based research, where patients can receive clinical trial services in many types of settings.
Consistently support the use of digital health technologies. Wearable devices and other advances can help make trials more convenient for participants by enabling remote collection of data from them in real time as they go about their daily lives. This convenience can promote diversity by reducing the number of clinic visits needed, making it possible for people to participate in trials whose income, work, or travel issues would prevent multiple in-person visits. Yet current FDA guidance lacks clarity on what evidence is needed to validate the use of digital health technologies. A modernized approach for qualifying digital health technologies is needed. Sponsors of new drug trials are currently encouraged to use the drug development tools pathway , which was not designed for digital health technologies and can be cumbersome and complicated for this use.
It also is not clear how digital health technologies will be reviewed when multiple FDA divisions or offices are involved. Providing greater clarity on the evidence required for validation and on cross-agency standards will support acceleration of the application of digital health technologies, further enabling community-based clinical trials.
By the end of this decade, we believe that community-based clinical trials will become the norm, not the outlier. To achieve this, all clinical trial stakeholders — including the FDA, drug developers, and investigators — must work together to foster a patient-centric clinical trial culture that embraces innovation and brings trials closer to potential participants. The result will be a win for everyone.
Robert Metcalf, Ph.D., is group vice president for clinical design, delivery and analytics, China and Japan medical, for Lilly. Jeffrey Francer, J.D., is Lilly’s vice president, head of global regulatory policy and strategy.
Have an opinion on this essay submit a letter to the editor here ., about the authors reprints, robert metcalf, jeffrey francer.
Clinical trials
STAT encourages you to share your voice. We welcome your commentary, criticism, and expertise on our subscriber-only platform, STAT+ Connect
To submit a correction request, please visit our Contact Us page .
According to the Realtor.com ® June housing data , the market stabilized as mortgage rates also stabilized in June due to better-than-expected CPI readings . While the median list price nationwide stayed the same as last year, homes continue to see a price increase on per square foot basis. The time a typical home spends on the market increased compared to last year, as the inventory of homes for sale continued to grow, but homes were still snapped up more quickly than pre-pandemic levels. Meanwhile, although sellers—who are often buyers themselves—may be a little more disgruntled this spring due to a slower market that is requiring more price-adjustment than sellers faced last spring, they are not delisting their homes at any higher rate than last year. Just 6.3% of listings were delisted in the middle of June and this rate has been relatively stable since February.
However, the total count of delistings has risen by 16.1% compared to the same time last year. How can the share of delistings remain relatively stable while the count grows so rapidly? The answer is that total inventory has also grown at about the same rate as delistings, so while a growing number of sellers have taken their homes off the market this spring, there has been a proportionally equal number of sellers who are keeping their homes on the market* .
There were 36.7% more homes actively for sale on a typical day in June compared with the same time in 2023, marking the eighth consecutive month of annual inventory growth. This is also an acceleration from May, which was up 35% year-over-year. In the eight consecutive months of increasing inventory, the rate of growth in each subsequent month has increased. While inventory this June is much improved compared with the previous three years, it is still down 32.4% compared with typical 2017 to 2019 levels. This is a slight improvement from last month’s 34.6% gap, as inventory continues to slowly grow toward normalcy.
In June, as in the previous four months , the growth in homes particularly priced in the $200,000 to $350,000 range outpaced all other price categories, as home inventory in this range grew by 50.0% compared with last year, surpassing even last month’s high 45.1% growth rate. This increase is again primarily fueled by a greater availability of smaller and more affordable homes in the South .
The total number of homes for sale, including homes that were under contract but not yet sold, increased by 22.4% compared with last year, growing on an annual basis for the seventh month in a row and eclipsing last month’s rate of 20.6%.
The number of homes under contract but not yet sold (pending listings) increased by 2.4% which is unchanged from last month’s rate of 2.4%. After reports of consumer price growth flattening in May , mortgage rates fell sharply in June on expectations that the Federal Reserve will cut rates at the end of the year. Back in April we predicted that the growth in pending listings would slow , and this materialized in both May but idled in June. However, with rates falling and inventory growing, it is possible that sales could accelerate slightly in June’s reported numbers, after they declined by 0.7% in May .
However, sellers continued to list their homes in higher numbers this June as newly listed homes were 6.3% above last year’s levels and higher than May’s figure of 5.9%. This marks the eighth month of increasing listing activity after a 17-month streak of decline. Two factors have made listings activity more sensitive to changes in mortgage rates. First, many sellers are themselves also homebuyers. Second, many potential sellers with a current mortgage have a rate well-below today’s market rate, with 87% of outstanding mortgage loans at a sub-6% rate . The decrease in mortgage rates seen in June likely contributed to an increased pace of growth in listing activity. We expect selling activity to continue to normalize as rates inch their way down over the next year.
The South and West Are Closest to Bridging the Inventory Gap
In June, all four regions saw active inventory grow over the previous year. The South saw listings grow by 48.9%, while inventory grew by 35.8% in the West, 21.5% in the Midwest, and only 12.5% in the Northeast. Compared with the typical June from 2017 to 2019 before the COVID-19 Pandemic, the South saw the smallest gap in inventory, down 17.2% compared with pre-pandemic levels, while the gap was 22.8% in the West, and much larger in the Midwest and Northeast, at 48.6% and 57.1%, respectively.
The inventory of homes for sale increased in all of the largest 50 metros compared with last year. Metros that saw the most inventory growth included Tampa (+93.1%), Orlando (+81.5%), and Denver (+77.9%).
Despite higher inventory growth compared with last year, most metros still had a lower level of inventory when compared with pre-pandemic years. Among the 50 largest metro areas, eight metros saw higher levels of inventory in June compared with typical 2017 to 2019 levels. This is down from 11 metros last month. The top metros that saw inventory surpass pre-pandemic levels were predominantly in the South and West and included Austin (+41.2%), Memphis (24.9%), and San Antonio (+24.0%).
The South saw newly listed homes increase the most compared with last year
Compared with June 2023, newly listed home inventory increased the most in the West, by 9.8%, whereas new inventory grew by 7.6% in the South, 2.1% in the Northeast, and 0.7% in the Midwest. The gap in newly listed homes compared with pre-pandemic 2017 to 2019 levels was also the lowest in the South, where newly listed homes were 14.9% below pre-pandemic levels. In comparison, they were down 30.2% in the West, 29.5% in the Midwest, and 32.7% in the Northeast.
In June, 41 of the 50 largest metros saw new listings increase over the previous year, up from 38 last month. However, two large metros saw more newly listed homes this June compared with the typical pace of new listings from June 2017 to 2019 before the pandemic: San Antonio (+8.9%) and Jacksonville (+11.2%). The metros that saw the largest growth in newly listed homes compared with last year included Seattle (+30.5%), San Jose (+26.5%), and San Antonio (+21.8%).
The typical home spent 45 days on the market this June, which is two days more than the same time last year and one more day than last month. June marks the third month in a row where homes spent more time on the market compared with the previous year as inventory continues to grow and home sales remain sluggish. However, the time a typical home spends on the market is more than a week (8 days) less than the average June from 2017 to 2019.
In the South, where the growth in home inventory has been the largest, the typical home spent five more days on the market in June compared with last year, while out West homes are staying on the market three days longer. However, in the Midwest (-1 day) and Northeast (-4 days), homes are still spending less time on the market than last year.
While all regions are still seeing time on the market below pre-pandemic levels, in the West, homes are spending only one day less on the market compared with the typical June from 2017 to 2019. Time on the market was eight days less than pre-pandemic levels in the South, 10 days less in the Midwest, and 15 days less in the Northeast.
Meanwhile, time on the market decreased compared with last year in 26 of the 50 largest metro areas this June, down from 30 markets last month. It decreased the most in San Jose, Chicago, and Providence (-9 days). Time on the market increased compared with last year in 22 of the 50 largest metros, including Phoenix (+14 days), Tampa (+8 days), and Jacksonville (+7 days). Four predominantly Western markets saw homes spend more time on the market than typical 2017 to 2019 pre-pandemic timing: Austin (+6 days), Portland (+4 days), and Oklahoma City (+1 day).
The national median list price continued to increase seasonally, to $445,000 in June compared with $440,000 in May, and the median list price remained stable compared with the same time last year, when it was also $445,000. However, when a change in the mix of inventory toward smaller homes is accounted for, the typical home listed this year has increased in asking price compared with last year. The median listing price per square foot increased by 3.4% in June compared with the same time last year. Moreover, the typical listed home price has grown by 39.1% compared with June 2019, while the price per square foot grew by 52.6%.
While the percentage of homes with price reductions increased from 14.1% in June of last year to 18.3% this year, the overall share of inventory is a little higher (+1.3pp) with the shares seen between June 2017 to June 2019.
In June, listing prices fell on a year-over-year basis in the South (-1.8%), where competitive home inventory has grown the most, but prices continued to increase in the Northeast (+5.6%), Midwest (3%), and West (+1.4%) compared with the same time last year. Controlling for the mix of homes on the market by looking at price-per-square-foot, prices in all regions showed greater growth rates of 2.6% to 7.2%. Among large metros, the median list price in Cleveland (+14.7%), Philadelphia (+11.3%), and Rochester (+9.3%) saw the biggest increases.
Meanwhile, all 50 large metropolitan areas have seen sizable price growth compared with homes listed before the pandemic. Compared with June 2019, the price per square foot growth rate in the largest 50 metros ranged from 24.4% to 81.9%. The markets where sellers saw the greatest increase in price per square foot included the New York metro area (+81.9% vs June 2019), Boston (+67.7%), and Tampa (+67.7%). Markets which saw the lowest return included San Jose (+24.4%), Baltimore (+24.6%), and New Orleans (+25.5%).
The share of price reductions was up compared with last year in the South (+5.1 percentage points), West (+4.5 percentage points), Midwest (+2.6 percentage points), and Northeast (+2.1 percentage points). Forty-seven of the 50 largest metros saw the share of price reductions increase compared with last June, up from 46 in May. Tampa saw the greatest increase (+10.9 percentage points), followed by Jacksonville (+9.7 percentage points), and Denver (+9.7 percentage points).
Midwest | 21.5% | 0.7% | 3.0% | 4.3% | -1 | 2.6 pp |
Northeast | 12.5% | 2.1% | 5.6% | 7.2% | -4 | 2.1 pp |
South | 48.9% | 7.6% | -1.8% | 2.6% | 5 | 5.1 pp |
West | 35.8% | 9.8% | 1.4% | 4.8% | 3 | 4.5 pp |
Midwest | -48.6% | -29.5% | 41.6% | 45.9% | -10 | -2.2 pp |
Northeast | -57.1% | -32.7% | 54.1% | 65.0% | -15 | -5.9 pp |
South | -17.2% | -14.9% | 35.1% | 53.2% | -8 | 4.2 pp |
West | -22.8% | -29.5% | 38.7% | 48.9% | -1 | 2.4 pp |
$425,000 | -2.9% | 1.7% | 30.8% | 52.6% | |
$550,000 | -5.2% | -2.1% | 49.1% | 62.4% | |
$369,900 | 1.3% | 0.8% | 8.8% | 24.6% | |
$301,900 | 0.7% | 1.3% | 13.1% | 26.5% | |
$879,000 | 1.4% | 5.9% | 46.5% | 67.7% | |
$299,900 | 8.0% | 7.2% | 34.9% | 46.6% | |
$441,170 | 0.0% | 3.5% | 23.1% | 56.5% | |
$399,900 | 4.5% | 5.8% | 19.4% | 32.4% | |
$379,900 | -2.6% | 3.9% | 33.3% | 51.3% | |
$285,000 | 14.7% | 14.4% | 35.8% | 32.4% | |
$400,000 | 0.1% | 5.0% | 23.3% | 53.5% | |
$459,000 | -3.0% | 0.5% | 27.8% | 45.1% | |
$639,000 | -6.0% | 1.4% | 24.1% | 46.7% | |
$275,000 | 1.9% | 2.3% | 5.0% | 26.0% | |
$449,900 | 3.6% | 12.5% | 47.6% | 64.5% | |
$372,900 | -1.4% | -0.1% | 15.5% | 38.3% | |
$352,495 | 0.7% | 3.9% | 25.9% | 56.4% | |
$424,000 | -3.4% | -0.4% | 36.2% | 54.1% | |
$429,000 | -5.2% | -2.2% | 32.0% | 47.9% | |
$484,988 | 6.6% | 7.6% | 49.3% | 56.2% | |
$1,249,000 | 6.8% | 6.5% | 51.4% | 55.1% | |
$342,952 | 5.5% | 2.1% | 19.5% | 41.4% | |
$349,000 | 6.7% | 2.0% | 51.7% | 63.8% | |
$535,000 | -11.5% | -8.4% | 33.8% | 47.9% | |
$400,000 | 5.3% | 5.3% | 42.9% | 43.0% | |
$460,000 | 0.0% | 2.2% | 28.6% | 36.0% | |
$575,000 | -2.7% | 3.9% | 52.1% | 66.7% | |
$335,000 | -2.9% | -2.5% | 15.5% | 25.5% | |
$789,000 | 5.3% | 8.6% | 34.9% | 81.9% | |
$335,000 | -4.3% | 0.3% | 30.9% | 44.7% | |
$445,000 | -3.1% | -0.1% | 39.1% | 55.1% | |
$395,000 | 11.3% | 9.0% | 36.3% | 54.8% | |
$539,900 | 0.0% | -0.2% | 42.1% | 55.3% | |
$259,900 | 8.3% | 10.8% | 30.0% | 31.1% | |
$625,000 | -2.3% | 2.0% | 30.5% | 41.5% | |
$599,000 | 8.9% | 8.8% | 55.6% | 47.7% | |
$470,000 | -1.9% | 3.3% | 25.0% | 52.7% | |
$475,000 | 7.4% | 5.9% | 41.8% | 58.1% | |
$615,000 | 6.0% | 5.3% | 46.5% | 61.9% | |
$299,900 | 9.3% | 10.4% | 30.4% | 39.7% | |
$679,000 | 0.0% | 4.2% | 35.5% | 40.3% | |
$349,000 | -5.0% | -2.2% | 18.3% | 39.3% | |
$1,048,944 | -4.0% | 5.5% | 45.7% | 64.2% | |
$998,000 | -13.2% | -6.3% | 4.0% | 28.5% | |
$1,441,979 | -3.7% | -1.3% | 22.7% | 24.4% | |
$795,000 | -3.6% | 0.3% | 29.3% | 48.5% | |
$314,900 | 8.8% | 5.7% | 34.1% | 32.8% | |
$425,000 | -4.5% | -0.7% | 49.1% | 67.7% | |
$399,000 | 1.0% | 5.1% | 35.3% | 45.2% | |
$632,004 | -1.9% | 6.5% | 30.3% | 59.8% |
58.6% | 13.5% | 38 | -1 | 22.5% | 9.0 pp | |
31.8% | 1.5% | 50 | 6 | 31.0% | -2.6 pp | |
29.4% | 2.1% | 31 | -5 | 15.0% | 4.2 pp | |
40.2% | 2.1% | 45 | 3 | 16.9% | 4.6 pp | |
23.1% | 6.9% | 25 | 1 | 15.1% | 3.0 pp | |
10.0% | 8.7% | 22 | -8 | 8.7% | 1.7 pp | |
49.3% | 13.9% | 36 | -2 | 21.0% | 8.4 pp | |
5.8% | 0.3% | 25 | -9 | 11.4% | 1.6 pp | |
30.4% | 9.5% | 29 | -1 | 14.5% | 4.2 pp | |
5.6% | 1.3% | 31 | -6 | 12.9% | 2.4 pp | |
32.3% | 3.9% | 25 | 3 | 20.1% | 5.6 pp | |
52.3% | 10.4% | 40 | 4 | 28.1% | 7.4 pp | |
77.9% | 7.6% | 30 | 2 | 29.8% | 9.7 pp | |
10.3% | -2.7% | 31 | 1 | 11.8% | -2.4 pp | |
6.4% | -1.7% | 17 | -1 | 6.4% | 0.5 pp | |
39.7% | 17.4% | 40 | 0 | 19.9% | 3.6 pp | |
28.8% | -6.2% | 35 | -1 | 22.9% | 5.8 pp | |
69.6% | 21.8% | 52 | 7 | 28.4% | 9.7 pp | |
23.9% | 5.1% | 44 | -6 | 15.9% | 3.9 pp | |
-29.5% | 15.5% | 38 | -6 | 18.4% | 4.8 pp | |
36.9% | 11.2% | 37 | -2 | 12.1% | 3.3 pp | |
28.7% | 6.3% | 31 | 2 | 16.7% | 3.6 pp | |
53.3% | 8.7% | 49 | 6 | 22.7% | 6.5 pp | |
67.7% | 12.7% | 67 | 5 | 18.2% | 5.5 pp | |
20.6% | -3.5% | 30 | 1 | 9.6% | 0.9 pp | |
21.8% | -6.3% | 28 | -4 | 13.7% | 2.4 pp | |
20.0% | 6.0% | 31 | -3 | 25.3% | 4.8 pp | |
28.6% | -0.4% | 61 | 3 | 21.8% | 1.5 pp | |
3.1% | -1.5% | 45 | -5 | 8.6% | 0.4 pp | |
38.7% | 11.4% | 45 | 0 | 22.1% | 6.8 pp | |
81.5% | 14.7% | 52 | 6 | 23.0% | 8.0 pp | |
10.8% | 1.6% | 38 | -7 | 13.1% | 1.5 pp | |
56.4% | 5.6% | 50 | 14 | 28.2% | 8.3 pp | |
14.1% | 4.8% | 44 | -3 | 16.2% | 1.9 pp | |
34.6% | -0.9% | 40 | 7 | 19.7% | 3.3 pp | |
22.9% | 9.3% | 23 | -9 | 9.5% | 3.1 pp | |
40.4% | 16.6% | 36 | -7 | 18.7% | 6.4 pp | |
39.5% | 5.7% | 36 | -2 | 11.3% | 4.2 pp | |
43.9% | 10.4% | 45 | 1 | 16.2% | 4.3 pp | |
3.1% | -2.1% | 17 | 5 | 3.9% | -3.5 pp | |
45.9% | 8.7% | 33 | 1 | 18.9% | 6.6 pp | |
48.6% | 21.8% | 50 | 4 | 26.3% | 3.2 pp | |
72.5% | 20.6% | 30 | -2 | 16.4% | 5.7 pp | |
39.5% | 11.3% | 29 | -3 | 13.3% | 3.2 pp | |
53.5% | 26.5% | 20 | -9 | 9.4% | 1.4 pp | |
61.9% | 30.5% | 24 | -5 | 15.6% | 3.1 pp | |
20.6% | 0.7% | 37 | -2 | 13.6% | 3.5 pp | |
93.1% | 18.1% | 53 | 8 | 29.8% | 10.9 pp | |
27.9% | 1.3% | 31 | 2 | 18.1% | 6.4 pp | |
27.2% | 8.3% | 29 | -3 | 12.9% | 3.8 pp |
* Note: Some metrics for the Las Vegas, Phoenix, and Rochester metro areas are under review and unavailable.
Realtor.com housing data as of June 2024. Listings include the active inventory of existing single-family homes and condos/townhomes/row homes/co-ops for the given level of geography on Realtor.com; new construction is excluded unless listed with an MLS that provides listing data to Realtor.com. Realtor.com data history goes back to July 2016. The 50 largest U.S. metropolitan areas as defined by the Office of Management and Budget (OMB-202003).
* Note that not all listing sources report sales of a home to Realtor.com, so in order to calculate an accurate estimate of delisted homes, we use a subset of counties where consistent sales of listings is reported. As such, our estimates of total inventory when calculating delistings will be different from our overall estimates of inventory. While different, the national trend in delisting activity should be more accurate than estimates based on a sample that includes sources that lack, or are inconsistent in, reporting sold listings.
Join our mailing list to receive the latest data and research.
Your browser does not support the <audio> element.
“C ERTAINLY, HERE is a possible introduction for your topic...” began a recent article in Surfaces and Interfaces , a scientific journal. Attentive readers might have wondered who exactly that bizarre opening line was addressing. They might also have wondered whether the ensuing article, on the topic of battery technology, was written by a human or a machine .
It is a question ever more readers of scientific papers are asking. Large language models ( LLM s) are now more than good enough to help write a scientific paper. They can breathe life into dense scientific prose and speed up the drafting process, especially for non-native English speakers. Such use also comes with risks: LLM s are particularly susceptible to reproducing biases, for example, and can churn out vast amounts of plausible nonsense. Just how widespread an issue this was, though, has been unclear.
In a preprint posted recently on arXiv, researchers based at the University of Tübingen in Germany and Northwestern University in America provide some clarity. Their research, which has not yet been peer-reviewed, suggests that at least one in ten new scientific papers contains material produced by an LLM . That means over 100,000 such papers will be published this year alone. And that is a lower bound. In some fields, such as computer science, over 20% of research abstracts are estimated to contain LLM -generated text. Among papers from Chinese computer scientists, the figure is one in three.
Spotting LLM -generated text is not easy. Researchers have typically relied on one of two methods: detection algorithms trained to identify the tell-tale rhythms of human prose, and a more straightforward hunt for suspicious words disproportionately favoured by LLM s, such as “pivotal” or “realm”. Both approaches rely on “ground truth” data: one pile of texts written by humans and one written by machines. These are surprisingly hard to collect: both human- and machine-generated text change over time, as languages evolve and models update. Moreover, researchers typically collect LLM text by prompting these models themselves, and the way they do so may be different from how scientists behave.
The latest research by Dmitry Kobak, at the University of Tübingen, and his colleagues, shows a third way, bypassing the need for ground-truth data altogether. The team’s method is inspired by demographic work on excess deaths, which allows mortality associated with an event to be ascertained by looking at differences between expected and observed death counts. Just as the excess-deaths method looks for abnormal death rates, their excess-vocabulary method looks for abnormal word use. Specifically, the researchers were looking for words that appeared in scientific abstracts with a significantly greater frequency than predicted by that in the existing literature (see chart 1). The corpus which they chose to analyse consisted of the abstracts of virtually all English-language papers available on PubMed, a search engine for biomedical research, published between January 2010 and March 2024, some 14.2m in all.
The researchers found that in most years, word usage was relatively stable: in no year from 2013-19 did a word increase in frequency beyond expectation by more than 1%. That changed in 2020, when “ SARS ”, “coronavirus”, “pandemic”, “disease”, “patients” and “severe” all exploded. (Covid-related words continued to merit abnormally high usage until 2022.)
By early 2024, about a year after LLM s like Chat GPT had become widely available, a different set of words took off. Of the 774 words whose use increased significantly between 2013 and 2024, 329 took off in the first three months of 2024. Fully 280 of these were related to style, rather than subject matter. Notable examples include: “delves”, “potential”, “intricate”, “meticulously”, “crucial”, “significant”, and “insights” (see chart 2).
The most likely reason for such increases, say the researchers, is help from LLM s. When they estimated the share of abstracts which used at least one of the excess words (omitting words which are widely used anyway), they found that at least 10% probably had LLM input. As PubMed indexes about 1.5m papers annually, that would mean that more than 150,000 papers per year are currently written with LLM assistance.
This seems to be more widespread in some fields than others. The researchers’ found that computer science had the most use, at over 20%, whereas ecology had the least, with a lower bound below 5%. There was also variation by geography: scientists from Taiwan, South Korea, Indonesia and China were the most frequent users, and those from Britain and New Zealand used them least (see chart 3). (Researchers from other English-speaking countries also deployed LLM s infrequently.) Different journals also yielded different results. Those in the Nature family, as well as other prestigious publications like Science and Cell , appear to have a low LLM- assistance rate (below 10%), while Sensors (a journal about, unimaginatively, sensors), exceeded 24%.
The excess-vocabulary method’s results are roughly consistent with those from older detection algorithms, which looked at smaller samples from more limited sources. For instance, in a preprint released in April 2024, a team at Stanford found that 17.5% of sentences in computer-science abstracts were likely to be LLM -generated. They also found a lower prevalence in Nature publications and mathematics papers ( LLM s are terrible at maths). The excess vocabulary identified also fits with existing lists of suspicious words.
Such results should not be overly surprising. Researchers routinely acknowledge the use of LLM s to write papers. In one survey of 1,600 researchers conducted in September 2023, over 25% told Nature they used LLM s to write manuscripts. The largest benefit identified by the interviewees, many of whom studied or used AI in their own work, was to help with editing and translation for those who did not have English as their first language. Faster and easier coding came joint second, together with the simplification of administrative tasks; summarising or trawling the scientific literature; and, tellingly, speeding up the writing of research manuscripts.
For all these benefits, using LLM s to write manuscripts is not without risks. Scientific papers rely on the precise communication of uncertainty, for example, which is an area where the capabilities of LLM s remain murky. Hallucination—whereby LLM s confidently assert fantasies—remains common, as does a tendency to regurgitate other people’s words, verbatim and without attribution.
Studies also indicate that LLM s preferentially cite other papers that are highly cited in a field, potentially reinforcing existing biases and limiting creativity. As algorithms, they can also not be listed as authors on a paper or held accountable for the errors they introduce. Perhaps most worrying, the speed at which LLM s can churn out prose risks flooding the scientific world with low-quality publications.
Academic policies on LLM use are in flux. Some journals ban it outright. Others have changed their minds. Up until November 2023, Science labelled all LLM text as plagiarism, saying: “Ultimately the product must come from—and be expressed by—the wonderful computers in our heads.” They have since amended their policy: LLM text is now permitted if detailed notes on how they were used are provided in the method section of papers, as well as in accompanying cover letters. Nature and Cell also allow its use, as long as it is acknowledged clearly.
How enforceable such policies will be is not clear. For now, no reliable method exists to flush out LLM prose. Even the excess-vocabulary method, though useful at spotting large-scale trends, cannot tell if a specific abstract had LLM input. And researchers need only avoid certain words to evade detection altogether. As the new preprint puts it, these are challenges that must be meticulously delved into. ■
Curious about the world? To enjoy our mind-expanding science coverage, sign up to Simply Science , our weekly subscriber-only newsletter.
This article appeared in the Science & technology section of the print edition under the headline “Scientists, et ai”
The race to prevent satellite armageddon.
Discover stories from this section and more in the list of contents
One Chilean hybrid has a spicy taste, with hints of clove
It would be fast enough to guide the hands of neurosurgeons
Even after a century of research, Barro Colorado in Panama continues to shed light on natural life
Those using the prosthetic can walk as fast as those with intact lower limbs
Small genetic mutations accumulated through inbreeding may have made them vulnerable to disease
Fears of a Russian nuclear weapon in orbit are inspiring new protective tech
It takes more than lip service to convince disabled job applicants to apply to your company. These research-backed practices can demonstrate that you’re a truly equitable employer.
How can companies do a better job of attracting disabled people to apply for jobs and convincing them that they truly are an equitable employer? And how can job candidates feel more comfortable disclosing a need for accommodation? The authors’ research over the last five years offers a number of paths forward for both sides. First, employers can move away from legalistic boilerplates and use more heartfelt language about their commitment to DEI. But they also need to back up their words with concrete evidence, such as a personal message from the CEO; testimonials from disabled employees; statistics on the hiring, promotion, accommodation fulfillment, and retention of disabled employees; or awards recognizing the company’s accomplishments in the DEI space. Their research also suggests job candidates emphasize their hard skills during interviews and delay the conversation about accommodations until they’ve built up more of a rapport with the hiring team.
Despite recent record employment gains for disabled employees in the U.S., the hiring of disabled people continues to be a pain point for both candidates and companies.
IMAGES
VIDEO
COMMENTS
Statistics articles from across Nature Portfolio. Statistics is the application of mathematical concepts to understanding and analysing large collections of data. A central tenet of statistics is ...
Article 05 July 2024 | Open Access. Research on bearing fault diagnosis based on improved genetic algorithm and BP neural network. Zenghua Chen, Lingjian Zhu & Gang Xiong
This article covers many statistical ideas essential to research statistical analysis. Sample size is explained through the concepts of statistical significance level and power. ... Finally, the most common statistics produced by these methods are explored. Keywords: statistical analysis, sample size, power, t-test, anova, chi-square ...
The "New Statistics" emphasizes effect sizes, confidence intervals, meta-analysis, and the use of Open Science practices. We present 3 specific ways in which a New Statistics approach can help improve scientific practice: by reducing over-confidence in small samples, by reducing confirmation bias, and by fostering more cautious judgments of consistency.
Statistics articles within Nature. Featured. Review Article | 02 August 2023. ... Research articles News Opinion Research Analysis Careers ...
A forthcoming article will be reporting on the follow-up of 8043 residents for mortality, 2000-2016; it found 16% deaths (1429) among the residents and "statistically significantly elevated lung cancer standardized mortality ratios (SMRs)" as well as for asbestosis and mesothelioma. ... Statistics, and Research, and Discussion community ...
Overview. Statistical Papers is a forum for presentation and critical assessment of statistical methods encouraging the discussion of methodological foundations and potential applications. The Journal stresses statistical methods that have broad applications, giving special attention to those relevant to the economic and social sciences.
June 12, 2024. This JAMA Guide to Statistics and Methods article explains the test-negative study design, an observational study design routinely used to estimate vaccine effectiveness, and examines its use in a study that estimated the performance of messenger RNA boosters against the Omicron variant. Research, Methods, Statistics Vaccination ...
In this article, we present ten research areas that could make statistics and data science more impactful on science and society. Focusing on these areas will help better transform data into knowledge, actionable insights and deliverables, and promote more collaboration with computer and other quantitative scientists and domain scientists ...
Browse Statistics news, research and analysis from The Conversation Menu ... Articles on Statistics. Displaying 1 - 20 of 256 articles. Khakimullin Aleksandr/Shutterstock June 12, 2024
An important requirement for validity of medical research is sound methodology and statistics, yet this is still often overlooked by medical researchers.1,2 Based on the experience of reviewing statistics in more than 1000 manuscripts submitted to The Lancet Group of journals over the past 3 years, this Correspondence provides guidance to commonly encountered statistical deficiencies in ...
Statistical Methods in Medical Research is a highly ranked, peer reviewed scholarly journal and is the leading vehicle for articles in all the main areas of medical statistics and therefore an essential reference for all medical statisticians. It is particularly useful for medical researchers dealing with data and provides a key resource for medical and statistical libraries, as well as ...
Journal of Probability and Statistics publishes papers on the theory and application of probability and statistics that consider new methods and approaches to their implementation, or report significant results ... Research Article. Open access. Bayesian Estimation of the Stress‐Strength Reliability Based on Generalized Order Statistics for ...
Although it may seem harmless, using statistics to "spin" results can prevent publication, undermine a published study, or lead to investigation and retraction. ... If the editors approve your study design, you'll receive a provisional acceptance for a future research article reporting the results. Preregistering is a great way to head ...
Estimating changepoints in extremal dependence, applied to aviation stock prices during COVID-19 pandemic. Arnab Hazra et al. Article | Published online: 3 Jul 2024. View all latest articles. Explore the current issue of Journal of Applied Statistics, Volume 51, Issue 9, 2024.
Biostatistics articles from across Nature Portfolio. Biostatistics is the application of statistical methods in studies in biology, and encompasses the design of experiments, the collection of ...
You can search the Library's research databases for peer-reviewed articles that use statistics to support a position or argument. You may have to try several different searches to find relevant articles. Peer-reviewed articles can be very valuable if you are struggling to locate statistics or data from a government site or database.
In the field of research, statistics is important for the following reasons: Reason 1: Statistics allows researchers to design studies such that the findings from the studies can be extrapolated to a larger population. Reason 2: Statistics allows researchers to perform hypothesis tests to determine if some claim about a new drug, new procedure ...
Research. Research Papers / Publications; Research Centers; Wharton Seminars / Conferences. Previous Statistics Seminars; Related Seminars; Programs. Undergraduate Program. Undergraduate Contact Information; Undergraduate Statistics Concentration; Undergraduate Statistics Minor; Business Analytics Joint Concentration; Undergraduate Course ...
Bad statistics may lead to bad research, and bad research may lead to unethical practice. Hence, an adequate knowledge of statistics and the appropriate use of statistical tests are important. An appropriate knowledge about the basic statistical methods will go a long way in improving the research designs and producing quality medical research ...
Journal of Survey Statistics and Methodology Sponsored by the ASA and American Association for Public Opinion Research, this journal's objective is to include cutting-edge scholarly articles on statistical and methodological issues for sample surveys, censuses, administrative record systems, and other related data.
Types of descriptive statistics. There are 3 main types of descriptive statistics: The distribution concerns the frequency of each value. The central tendency concerns the averages of the values. The variability or dispersion concerns how spread out the values are. You can apply these to assess only one variable at a time, in univariate ...
CFViSA integrates two omics data analysis pipelines (microbiome and transcriptome analysis) and an extensive array of 79 analysis tools spanning simple sequence processing, visualization, and statistics available for various omics data, including microbiome and transcriptome data.
The report, "The Nation's Data at Risk," highlights the threats facing statistics produced across the federal government, including data on education, health, crime and demographic trends.
The following are the general steps for statistical analysis: (1) formulate a hypothesis, (2) select an appropriate statistical test, (3) conduct a power analysis, (4) prepare data for analysis, (5) start with descriptive statistics, (6) check assumptions of tests, (7) run the analysis, (8) examine the statistical model, (9) report the results ...
Research sponsors and the Food and Drug Administration can respond to this challenge by continuing to support community-based clinical trials. But the regulatory framework that governs these and ...
June 2024 Regional Statistics vs. Pre-Pandemic 2017-19. Region: Active Listing Count vs Pre-Pandemic: ... Join our mailing list to receive the latest data and research.
They might also have wondered whether the ensuing article, on the topic of battery technology, was written by a human or a machine. It is a question ever more readers of scientific papers are asking.
The authors' research over the last five years offers a number of paths forward for both sides. First, employers can move away from legalistic boilerplates and use more heartfelt language about ...