Interval-Censored Survival Analysis

Suppose a factory inspects its 100 industrial machines every 3 months. One machine was working normally at the 3-month inspection but was found broken at the 6-month inspection. So when did it fail?

We can’t know the exact date of the failure. All we can confirm is that it happened at some moment after the 3-month inspection and before, or by, the 6-month inspection.

Data like this, where we know that an event happened but not exactly when, and can only identify the time interval in which the event occurred, are called interval-censored data.

Ordinary Kaplan–Meier survival analysis mainly deals with cases where the event time is recorded exactly or where observation ended before the event happened. Interval-censored survival analysis, in contrast, uses the information that the event time lies within a particular interval.

In this lesson we’ll look at the idea of interval censoring, how it differs from ordinary censoring, how survival probabilities are estimated, a worked analysis of real equipment failure data, and how to interpret the results and what to watch out for.

For example, suppose a machine was working normally at the 4-month inspection after installation and was found broken at the 8-month inspection. The machine’s failure time lies in the interval greater than 4 months and at most 8 months. As a formula:

Here T is the actual time of failure.

What the researcher knows is not the exact failure time but the interval that contains it. If you simply record the failure time in such data as 8 months, the failure can end up recorded as happening later than it really did.

Conversely, assuming that the failure actually happened at 6 months, the midpoint between the two inspections, also adds information that has no basis. Interval-censored survival analysis estimates the survival function from the observed time intervals without such arbitrary time imputation.

The main types are right censoring, left censoring and interval censoring.

Right censoring is the most commonly handled type. For example, if a machine was still working at the last inspection at 12 months, we don’t know when it will fail after that, but we do know that it didn’t fail for at least 12 months.

Left censoring is when the event is found to have already happened at the first inspection, but its exact time before that is unknown. Interval censoring is when we know both the last time the normal state was confirmed and the first time the event was confirmed.

All three types share the fact that the actual event time can’t be fully known, but the time information available differs, so the analysis method has to be chosen accordingly.

The lower bound is the last time the subject was confirmed to be in the pre-event state, and the upper bound is the first time the event was confirmed to have occurred. For example, suppose six machines had the following inspection results.

An example to illustrate the data structure.

Machine A failed between 3 and 6 months. Machine E was still working at the 12-month inspection, so it is treated as right-censored. Machine F was working at 6 months and found broken at 12 months. If there was no 9-month inspection record, its failure interval can be recorded as (6, 12] months.

Interval-censored survival analysis can handle these different observation intervals together. However, not every method supports every kind of censoring in the same way, so check what your statistical model and software can do.

All machines were working normally when installed, and scheduled inspections were carried out every 3 months after that. When a machine was first found broken, its failure interval was identified using the last inspection at which it had been working. The observation period was 12 months in total.

Equipment inspection results

In these data, no machine has an exactly known failure time. For the 55 machines that failed we know only the time interval of failure, and the other 45 were still working at 12 months. The research question is:

Interval-censored survival analysis can be applied to answer this question.

Instead, the survival function is estimated using, for each observation, the probability that the event time falls within its given interval. A well-known nonparametric method is the Turnbull estimator.

The Turnbull estimator is a nonparametric maximum likelihood method that estimates the distribution of survival times from the observed event intervals and censoring information. The likelihood contribution of an interval-censored observation can be written in terms of the survival function S(t) as follows.

Here L is the last time the subject was confirmed event-free, and R is the first time the event was confirmed. For example, if a machine’s failure interval is (3, 6] months, this observation provides information equal to the difference between the survival probabilities at 3 and 6 months.

Meanwhile, a machine that was still working at 12 months provides information equal to S(12), the probability of surviving beyond 12 months. The Turnbull estimator pools the information from these observations to estimate the survival function.

In general interval-censored data, however, the survival probability at every exact event time isn’t identified. Especially when observation intervals overlap or are wide, parts of the survival function may not be uniquely estimated.

We also assumed that no inspections were missed during the observation period and that machines still working at the end were right-censored at 12 months. So in these simple data, the survival probability up to each inspection time can be calculated as follows.

Interval-censored survival analysis results

The lines between inspection times in the graph are drawn only to connect the points. These data alone can’t tell us exactly what path the survival probability took within each interval.

The survival probability at 3 months is 90%. This means the estimated probability that an installed machine works without failure through 3 months is 90%. At 6 months the survival probability dropped to 75%, so the estimated probability of working without failure through 6 months is 75%.

It fell to 55% at 9 months and 45% at 12 months. This shows failures accumulating over time. But we can’t know exactly on what date each of the 15 machines that failed between 3 and 6 months broke down.

So the survival probability at 4 or 5 months can’t be determined exactly without additional information.

For example, 90 machines were still working at 3 months. Of these, 15 failed between 3 and 6 months. So the conditional probability that a machine working at 3 months fails during the next 3 months is:

The results for each interval are:

Conditional probabilities assuming equal inspection intervals and complete observation.

In these results the highest interval failure probability occurred at 6–9 months. So about 26.7% of the machines still working at 6 months failed during the following 3 months. Information like this can be useful when reviewing inspection schedules or preventive maintenance plans.

However, the fact that failure probability was high in a particular period doesn’t by itself let us conclude that some specific cause of failure operates at that time.

In the results above, the survival probability was 55% at 9 months and 45% at 12 months. So we can conclude that the median failure time lies between 9 and 12 months.

This means that the time by which half of all machines fail lies in the interval greater than 9 months and at most 12 months. But these data alone can’t determine that the median failure time is exactly 10.5 months. 10.5 months is just the midpoint between two inspection times, not a direct observation of when failures actually happened.

Using a parametric model that assumes a particular survival-time distribution lets you estimate the median survival time inside the interval. The result, however, can depend on the chosen distribution and model assumptions. So if you’ve done only a nonparametric analysis, it may be more appropriate to report the interval that contains the median survival time.

The Turnbull estimator is the best-known nonparametric estimator for interval-censored data. Parametric survival models estimate the survival function by assuming that event times follow a particular probability distribution, for example the Weibull or log-normal distribution.

Parametric models can be used to estimate survival probabilities inside observation intervals or to analyze the relationship between predictors and survival time, but if the distributional assumption is inappropriate the results can be biased. Using a regression model that supports interval censoring also lets you take several variables into account together, such as the machine’s manufacturer, operating environment and part type.

But which method applies depends on how precisely the event times were observed in the data.

For example, if sensors on the machines record the exact date and time of each failure, you can use the Kaplan–Meier estimator. If only the total number of failures each month has been summarized, you can consider an interval-based life-table analysis.

If each machine has its own record of the last working inspection and the first failed inspection, and the actual failure time is unknown, interval-censored survival analysis is the right fit. It matters especially that each observation can carry its own event interval when inspection intervals differ between machines or some inspections were missed.

Inspection times may be related to the risk of the event.

If only machines that are more likely to fail are inspected more often, the inspection interval may be related to the machine’s condition. If this observation process isn’t properly reflected in the model, estimates of the survival-time distribution can be biased.

Don’t impute event times arbitrarily.

Replacing the actual failure time with the midpoint between the last working inspection and the first confirmed failure can give the time information a false sense of certainty. Midpoint imputation can be used as an approximation when needed, but the possible bias should be examined, and methods that reflect interval censoring directly are generally more appropriate.

The wider the observation intervals, the more uncertain the estimates can be.

Data from weekly inspections and data from inspections every 6 months provide different levels of information about event times. In general, the wider the inspection interval, the harder it is to estimate the event time precisely.

Distinguish early failures from right censoring.

A machine already found broken at the first inspection should be recorded as having had the event before the first inspection. Conversely, a machine still working at the last inspection is right-censored data whose subsequent failure time wasn’t observed.

Be careful with prediction and extrapolation.

If the observation period is 12 months, those data alone can’t adequately tell you the survival probability at 24 or 36 months. A parametric model can predict survival probabilities at later times, but that depends on the assumed distribution.

For the industrial equipment failure example above, a report could read as follows.

In an actual analysis, the estimator used and the distribution of the observation intervals should be reported clearly. Where possible, it’s also good to present confidence intervals for the survival probabilities and the number of observations per interval to show the uncertainty of the estimates.

Medical and health research

In studies that examine patients every 3 or 6 months, you can analyze cases where disease recurrence or a particular abnormal condition occurred between examinations.

Manufacturing and equipment management

When scheduled inspections are used to check whether machines or parts have failed, the time intervals of failure can be used to analyze product reliability.

Education research

When a student’s particular academic status is first identified in an assessment given every semester, the time it arose can be recorded as an interval.

Social science research

It can be used when events such as job loss or a change in housing status occur between two waves of a regular panel survey.

Finance and credit risk analysis

It can be applied when a particular risk status is first identified in a periodic credit review but the exact time the change actually happened is unknown.

But you need to distinguish whether what you actually observed is the event itself or a state found some time after the event occurred. If an event can occur temporarily and disappear before the next inspection, a separate observation model may be needed.

For example, if a machine was working normally at the 3-month inspection and found broken at the 6-month inspection, the actual failure time lies in the interval greater than 3 months and at most 6 months. With such data, it’s appropriate to use the observed time interval itself in the analysis rather than arbitrarily setting the event time to 6 months or 4.5 months.

The best-known method, the Turnbull estimator, estimates the survival function nonparametrically from the event intervals and censoring information. Depending on the characteristics of the data and the research goal, parametric survival models or interval-censored regression models can also be used.

However, if inspection intervals are wide or inspection times are related to the machine’s condition, the estimates can be uncertain or biased, so the observation process and the analysis assumptions should be reviewed together.