StandardDeviation.java

  1. /*
  2.  * Licensed to the Apache Software Foundation (ASF) under one or more
  3.  * contributor license agreements.  See the NOTICE file distributed with
  4.  * this work for additional information regarding copyright ownership.
  5.  * The ASF licenses this file to You under the Apache License, Version 2.0
  6.  * (the "License"); you may not use this file except in compliance with
  7.  * the License.  You may obtain a copy of the License at
  8.  *
  9.  *      http://www.apache.org/licenses/LICENSE-2.0
  10.  *
  11.  * Unless required by applicable law or agreed to in writing, software
  12.  * distributed under the License is distributed on an "AS IS" BASIS,
  13.  * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  14.  * See the License for the specific language governing permissions and
  15.  * limitations under the License.
  16.  */
  17. package org.apache.commons.statistics.descriptive;

  19. /**
  20.  * Computes the standard deviation of the available values. The default implementations uses
  21.  * the following definition of the <em>sample standard deviation</em>:
  22.  *
  23.  * <p>\[ \sqrt{ \tfrac{1}{n-1} \sum_{i=1}^n (x_i-\overline{x})^2 } \]
  24.  *
  25.  * <p>where \( \overline{x} \) is the sample mean, and \( n \) is the number of samples.
  26.  *
  27.  * <ul>
  28.  *   <li>The result is {@code NaN} if no values are added.
  29.  *   <li>The result is {@code NaN} if any of the values is {@code NaN} or infinite.
  30.  *   <li>The result is {@code NaN} if the sum of the squared deviations from the mean is infinite.
  31.  *   <li>The result is zero if there is one finite value in the data set.
  32.  * </ul>
  33.  *
  34.  * <p>The use of the term \( n − 1 \) is called Bessel's correction. Omitting the square root,
  35.  * this provides an unbiased estimator of the variance of a hypothetical infinite population. If the
  36.  * {@link #setBiased(boolean) biased} option is enabled the normalisation factor is
  37.  * changed to \( \frac{1}{n} \) for a biased estimator of the <em>sample variance</em>.
  38.  * Note however that square root is a concave function and thus introduces negative bias
  39.  * (by Jensen's inequality), which depends on the distribution, and thus the corrected sample
  40.  * standard deviation (using Bessel's correction) is less biased, but still biased.
  41.  *
  42.  * <p>The {@link #accept(double)} method uses a recursive updating algorithm based on West's
  43.  * algorithm (see Chan and Lewis (1979)).
  44.  *
  45.  * <p>The {@link #of(double...)} method uses the corrected two-pass algorithm from
  46.  * Chan <i>et al</i>, (1983).
  47.  *
  48.  * <p>Note that adding values using {@link #accept(double) accept} and then executing
  49.  * {@link #getAsDouble() getAsDouble} will
  50.  * sometimes give a different, less accurate, result than executing
  51.  * {@link #of(double...) of} with the full array of values. The former approach
  52.  * should only be used when the full array of values is not available.
  53.  *
  54.  * <p>Supports up to 2<sup>63</sup> (exclusive) observations.
  55.  * This implementation does not check for overflow of the count.
  56.  *
  57.  * <p>This class is designed to work with (though does not require)
  58.  * {@linkplain java.util.stream streams}.
  59.  *
  60.  * <p><strong>Note that this instance is not synchronized.</strong> If
  61.  * multiple threads access an instance of this class concurrently, and at least
  62.  * one of the threads invokes the {@link java.util.function.DoubleConsumer#accept(double) accept} or
  63.  * {@link StatisticAccumulator#combine(StatisticResult) combine} method, it must be synchronized externally.
  64.  *
  65.  * <p>However, it is safe to use {@link java.util.function.DoubleConsumer#accept(double) accept}
  66.  * and {@link StatisticAccumulator#combine(StatisticResult) combine}
  67.  * as {@code accumulator} and {@code combiner} functions of
  68.  * {@link java.util.stream.Collector Collector} on a parallel stream,
  69.  * because the parallel instance of {@link java.util.stream.Stream#collect Stream.collect()}
  70.  * provides the necessary partitioning, isolation, and merging of results for
  71.  * safe and efficient parallel execution.
  72.  *
  73.  * <p>References:
  74.  * <ul>
  75.  *   <li>Chan and Lewis (1979)
  76.  *       Computing standard deviations: accuracy.
  77.  *       Communications of the ACM, 22, 526-531.
  78.  *       <a href="http://doi.acm.org/10.1145/359146.359152">doi: 10.1145/359146.359152</a>
  79.  *   <li>Chan, Golub and Levesque (1983)
  80.  *       Algorithms for Computing the Sample Variance: Analysis and Recommendations.
  81.  *       American Statistician, 37, 242-247.
  82.  *       <a href="https://doi.org/10.2307/2683386">doi: 10.2307/2683386</a>
  83.  * </ul>
  84.  *
  85.  * @see <a href="https://en.wikipedia.org/wiki/Standard_deviation">Standard deviation (Wikipedia)</a>
  86.  * @see <a href="https://en.wikipedia.org/wiki/Bessel%27s_correction">Bessel&#39;s correction</a>
  87.  * @see <a href="https://en.wikipedia.org/wiki/Jensen%27s_inequality">Jensen&#39;s inequality</a>
  88.  * @see Variance
  89.  * @since 1.1
  90.  */
  91. public final class StandardDeviation implements DoubleStatistic, StatisticAccumulator<StandardDeviation> {

  93.     /**
  94.      * An instance of {@link SumOfSquaredDeviations}, which is used to
  95.      * compute the standard deviation.
  96.      */
  97.     private final SumOfSquaredDeviations ss;

  99.     /** Flag to control if the statistic is biased, or should use a bias correction. */
  100.     private boolean biased;

  102.     /**
  103.      * Create an instance.
  104.      */
  105.     private StandardDeviation() {
  106.         this(new SumOfSquaredDeviations());
  107.     }

  109.     /**
  110.      * Creates an instance with the sum of squared deviations from the mean.
  111.      *
  112.      * @param ss Sum of squared deviations.
  113.      */
  114.     StandardDeviation(SumOfSquaredDeviations ss) {
  115.         this.ss = ss;
  116.     }

  118.     /**
  119.      * Creates an instance.
  120.      *
  121.      * <p>The initial result is {@code NaN}.
  122.      *
  123.      * @return {@code StandardDeviation} instance.
  124.      */
  125.     public static StandardDeviation create() {
  126.         return new StandardDeviation();
  127.     }

  129.     /**
  130.      * Returns an instance populated using the input {@code values}.
  131.      *
  132.      * <p>Note: {@code StandardDeviation} computed using {@link #accept(double) accept} may be
  133.      * different from this standard deviation.
  134.      *
  135.      * <p>See {@link StandardDeviation} for details on the computing algorithm.
  136.      *
  137.      * @param values Values.
  138.      * @return {@code StandardDeviation} instance.
  139.      */
  140.     public static StandardDeviation of(double... values) {
  141.         return new StandardDeviation(SumOfSquaredDeviations.of(values));
  142.     }

  144.     /**
  145.      * Updates the state of the statistic to reflect the addition of {@code value}.
  146.      *
  147.      * @param value Value.
  148.      */
  149.     @Override
  150.     public void accept(double value) {
  151.         ss.accept(value);
  152.     }

  154.     /**
  155.      * Gets the standard deviation of all input values.
  156.      *
  157.      * <p>When no values have been added, the result is {@code NaN}.
  158.      *
  159.      * @return standard deviation of all values.
  160.      */
  161.     @Override
  162.     public double getAsDouble() {
  163.         // This method checks the sum of squared is finite
  164.         // to provide a consistent NaN when the computation is not possible.
  165.         // Note: The SS checks for n=0 and returns NaN.
  166.         final double m2 = ss.getSumOfSquaredDeviations();
  167.         if (!Double.isFinite(m2)) {
  168.             return Double.NaN;
  169.         }
  170.         final long n = ss.n;
  171.         // Avoid a divide by zero
  172.         if (n == 1) {
  173.             return 0;
  174.         }
  175.         return biased ? Math.sqrt(m2 / n) : Math.sqrt(m2 / (n - 1));
  176.     }

  178.     @Override
  179.     public StandardDeviation combine(StandardDeviation other) {
  180.         ss.combine(other.ss);
  181.         return this;
  182.     }

  184.     /**
  185.      * Sets the value of the biased flag. The default value is {@code false}. The bias
  186.      * term refers to the computation of the variance; the standard deviation is returned
  187.      * as the square root of the biased or unbiased <em>sample variance</em>. For further
  188.      * details see {@link Variance#setBiased(boolean) Variance.setBiased}.
  189.      *
  190.      * <p>This flag only controls the final computation of the statistic. The value of
  191.      * this flag will not affect compatibility between instances during a
  192.      * {@link #combine(StandardDeviation) combine} operation.
  193.      *
  194.      * @param v Value.
  195.      * @return {@code this} instance
  196.      * @see Variance#setBiased(boolean)
  197.      */
  198.     public StandardDeviation setBiased(boolean v) {
  199.         biased = v;
  200.         return this;
  201.     }
  202. }
Created with JaCoCo 0.8.12.202403310830Code Coverage Report for Apache Commons Statistics 1.1