TTest
Details and Options
- TTest tests the null hypothesis
against the alternative hypothesis 
: -
data 
{data1,data2} 
- where μi is the population mean for datai.
- By default, a probability value or
-value is returned. - A small
-value suggests that it is unlikely that 
is true. - The data in dspec can be univariate {x1,x2,…} or multivariate {{x1,y1,…},{x2,y2,…},…}.
- The argument μ0 can be a real number or a real vector with length equal to the dimension of the data.
- TTest assumes that the data is normally distributed but is fairly robust to this assumption. TTest also assumes that the samples are independent in the two sample cases.
- TTest[dspec,μ0,"HypothesisTestData"] returns a HypothesisTestData object htd that can be used to extract additional test results and properties using the form htd["property"].
- TTest[dspec,μ0,"property"] can be used to directly give the value of "property".
- Properties related to the reporting of test results include:
-
"DegreesOfFreedom" the degrees of freedom used in a test "PValue" list of 
-values"PValueTable" formatted table of 
-values"ShortTestConclusion" a short description of the conclusion of a test "TestConclusion" a description of the conclusion of a test "TestData" list of pairs of test statistics and 
-values"TestDataTable" formatted table of 
-values and test statistics"TestStatistic" list of test statistics "TestStatisticTable" formatted table of test statistics - For univariate samples, TTest performs a Student
test. The test statistic is assumed to follow a StudentTDistribution[df]. - For multivariate samples, TTest performs Hotelling's
test. The test statistic is assumed to follow a HotellingTSquareDistribution[p,df] where p is the dimension of data. - The degrees of freedom df, used to specify the distribution of the test statistic, depend on the sample size, number of samples, and in the case of two univariate samples, the results of a test for equal variances.
- The following options can be used:
-
AlternativeHypothesis "Unequal" the inequality for the alternative hypothesis SignificanceLevel 0.05 cutoff for diagnostics and reporting VerifyTestAssumptions Automatic what assumptions to verify - For the TTest, a cutoff
is chosen such that 
is rejected only if 
. The value of 
used for the "TestConclusion" and "ShortTestConclusion" properties is controlled by the SignificanceLevel option. This value 
is also used in diagnostic tests of assumptions, including tests for normality, equal variance, and symmetry. By default, 
is set to 0.05. - Named settings for VerifyTestAssumptions in TTest include:
-
"Normality" verify that all data is normally distributed "EqualVariance" verify that data1 and data2 have equal variance
Examples
open all close allBasic Examples (3)
Test whether the mean of a population is zero:
data = RandomVariate[NormalDistribution[0.05, 1], 10^4];TTest[data]TTest[data, Automatic, "TestDataTable"]Test whether the means of two populations differ by 2:
BlockRandom[SeedRandom[1];data1 = RandomVariate[NormalDistribution[1.85, 1], 1000];
data2 = RandomVariate[NormalDistribution[0, 1], 1000]];
Mean[data1] - Mean[data2]SmoothHistogram[{data1, data2 + 2}]At the 0.05 level, 
is significantly different from 2:
TTest[{data1, data2}, 2]Compare the locations of multivariate populations:
BlockRandom[SeedRandom[1];data1 = RandomVariate[MultinormalDistribution[{1, 2}, IdentityMatrix[2]], 1000];
data2 = RandomVariate[MultinormalDistribution[{0, 0}, IdentityMatrix[2]], 1000]];
Mean[data1] - Mean[data2]Histogram3D[{data1, Transpose[Transpose[data2] + {1, 2}]}]At the 0.05 level, 
is not significantly different from {1,2}:
TTest[{data1, data2}, {1, 2}]Scope (13)
Testing (10)
SeedRandom[1];
data1 = RandomVariate[NormalDistribution[0, 1], 500];
data2 = RandomVariate[NormalDistribution[3, 1], 500];The 
-values are typically large when the mean is close to 
:
TTest[data1]The 
-values are typically small when the location is far from 
:
TTest[data2]Using Automatic is equivalent to testing for a mean of zero:
data = RandomVariate[NormalDistribution[0, 1], 500];TTest[data, 0]TTest[data, Automatic]
SeedRandom[1];
data1 = RandomVariate[NormalDistribution[3, 1], 500];
data2 = RandomVariate[NormalDistribution[0, 1], 200];The 
-values are typically large when the mean is close to 
:
TTest[data1, 3]The 
-values are typically small when the location is far from 
:
TTest[data2, 3]Test whether the mean vector of a multivariate dataset is the zero vector:
SeedRandom[1];data = RandomVariate[MultinormalDistribution[{.1, 0, -.05, 0}, IdentityMatrix[4]], 10^3];TTest[data]Alternatively, test against {0.1,0,–0.05,0}:
TTest[data, {0.1, 0, -.05, 0}]
SeedRandom[1];data1 = RandomVariate[NormalDistribution[0, 1], 100];
data2 = RandomVariate[NormalDistribution[1, 1], 150];
data3 = RandomVariate[NormalDistribution[0, 2], 130];The 
-values are generally small when the locations are not equal:
TTest[{data1, data2}, 0]The 
-values are generally large when the locations are equal:
TTest[{data1, data3}, 0]
SeedRandom[1];data1 = RandomVariate[NormalDistribution[3, 1], 135];
data2 = RandomVariate[NormalDistribution[0, 1], 100];The order of the datasets affects the test results:
TTest[{data1, data2}, 3]TTest[{data2, data1}, 3]Test whether the mean difference vector of two multivariate datasets is the zero vector:
SeedRandom[1];data1 = RandomVariate[MultinormalDistribution[{.5, 0, -.5, 0}, IdentityMatrix[4]], 122];data2 = RandomVariate[MultinormalDistribution[{-.5, 0, .5, 0}, IdentityMatrix[4]], 135];TTest[{data1, data2}]Alternatively, test against {1,0,–1,0}:
TTest[{data1, data2}, {1, 0, -1, 0}]Create a HypothesisTestData object for repeated property extraction:
SeedRandom[1];data = RandomVariate[NormalDistribution[], {2, 10^4}];ℋ = TTest[data, 0, "HypothesisTestData"];The properties available for extraction:
ℋ["Properties"]Extract some properties from a HypothesisTestData object:
SeedRandom[2];data = RandomVariate[NormalDistribution[], {2, 10^4}];ℋ = TTest[data, 0, "HypothesisTestData"];The 
-value, test statistic, and degrees of freedom:
ℋ["PValue"]ℋ["TestStatistic"]ℋ["DegreesOfFreedom"]Extract any number of properties simultaneously:
SeedRandom[2];data = RandomVariate[NormalDistribution[], {2, 10^4}];ℋ = TTest[data, 0, "HypothesisTestData"];The 
-value, test statistic, and degrees of freedom:
ℋ["PValue", "TestStatistic", "DegreesOfFreedom"]Reporting (3)
SeedRandom[1];data = RandomVariate[NormalDistribution[], {2, 20}];ℋ = TTest[data, 0, "HypothesisTestData"];ℋ["TestDataTable"]Retrieve the entries from a test table for customized reporting:
SeedRandom[1];data = RandomVariate[NormalDistribution[], {1000, 25}];res = Table[TTest[v, 0, "TestData", VerifyTestAssumptions -> None], {v, data}];ListPlot[res, FrameLabel -> {"T", "p-value"}, Frame -> True, PlotRange -> All]Tabulate 
-values or test statistics:
SeedRandom[1];data = RandomVariate[NormalDistribution[], {2, 10^2}];ℋ = TTest[data, 0, "HypothesisTestData"];ℋ["PValueTable"]
ℋ["PValue"]ℋ["TestStatisticTable"]The test statistic from the table:
ℋ["TestStatistic"]Options (11)
AlternativeHypothesis (3)
A two-sided test is performed by default:
SeedRandom[1];
data = RandomVariate[NormalDistribution[], 100];
TTest[data, 0, AlternativeHypothesis -> "Unequal"]TTest[data, 0, AlternativeHypothesis -> Automatic]Perform a two-sided test or a one-sided alternative:
SeedRandom[1];
data = RandomVariate[NormalDistribution[], 100];
TTest[data, 0, AlternativeHypothesis -> "Unequal"]
TTest[data, 0, AlternativeHypothesis -> "Less"]
TTest[data, 0, AlternativeHypothesis -> "Greater"]Perform tests with one-sided alternatives when 
is given:
SeedRandom[1];
data1 = RandomVariate[NormalDistribution[2.9, 1], 1000];
data2 = RandomVariate[NormalDistribution[0, 1], 1000];Mean[data1] - Mean[data2]
TTest[{data1, data2}, 3, AlternativeHypothesis -> "Less"]
TTest[{data1, data2}, 2.9, AlternativeHypothesis -> "Less"]SignificanceLevel (2)
Set the significance level for diagnostic tests:
data = BlockRandom[SeedRandom[2];RandomVariate[StudentTDistribution[3], 50]];TTest[data, 0, SignificanceLevel -> .0001]TTest[data, 0]
The significance level is also used for "TestConclusion" and "ShortTestConclusion":
BlockRandom[SeedRandom[1];data = RandomVariate[NormalDistribution[0, 1], 100]];Mean[data]ℋ1 = TTest[data, .2, "HypothesisTestData", SignificanceLevel -> .1];ℋ2 = TTest[data, .2, "HypothesisTestData", SignificanceLevel -> .005];ℋ1["TestConclusion"]//TraditionalFormℋ2["TestConclusion"]//TraditionalFormℋ1["ShortTestConclusion"]ℋ2["ShortTestConclusion"]VerifyTestAssumptions (6)
By default, normality and equal variance are tested:
data1 = RandomVariate[NormalDistribution[0, 1], 100];
data2 = RandomVariate[NormalDistribution[0, 2], 100];TTest[{data1, data2}, 0, VerifyTestAssumptions -> Automatic]If assumptions are not checked, some test results may differ:
TTest[{data1, data2}, 0, VerifyTestAssumptions -> None]Diagnostics can be controlled as a group using All or None:
data1 = RandomVariate[NormalDistribution[0, 1], 100];
data2 = RandomVariate[NormalDistribution[0, 2], 100];TTest[{data1, data2}, 0, VerifyTestAssumptions -> All]TTest[{data1, data2}, 0, VerifyTestAssumptions -> None]Diagnostics can be controlled independently:
data1 = RandomVariate[NormalDistribution[0, 1], 100];
data2 = RandomVariate[NormalDistribution[0, 2], 100];Assume normality but check for equal variances:
TTest[{data1, data2}, 0, VerifyTestAssumptions -> "EqualVariance"]TTest[{data1, data2}, 0, VerifyTestAssumptions -> "Normality"]Set the equal variance assumption to False:
TTest[{data1, data2}, 0, VerifyTestAssumptions -> "EqualVariance" -> False]Unlisted assumptions are not tested:
data = RandomVariate[CauchyDistribution[0, 1], 100];TTest[data, 0, VerifyTestAssumptions -> "EqualVariance"]The result is the same but a warning is issued:
TTest[data, 0, VerifyTestAssumptions -> All]
Bypassing diagnostic tests can save compute time:
data = RandomVariate[MultinormalDistribution[{0, 0, 0}, IdentityMatrix[3]], 10 ^ 4];TTest[data, Automatic, "TestDataTable", VerifyTestAssumptions -> All]//AbsoluteTimingTTest[data, Automatic, "TestDataTable", VerifyTestAssumptions -> None]//AbsoluteTimingIt is often useful to bypass diagnostic tests for simulation purposes:
data = RandomVariate[NormalDistribution[], {1000, 100}];AbsoluteTiming[T = Quiet@TTest[#, Automatic, "TestStatistic"]& /@ data;]The assumptions of the test hold by design, so a great deal of time can be saved:
AbsoluteTiming[T2 = Quiet@TTest[#, Automatic, "TestStatistic", VerifyTestAssumptions -> None]& /@ data;]SmoothHistogram[{T, T2}]Applications (4)
Test whether the means of some populations are equal:
SeedRandom[1];data1 = RandomVariate[NormalDistribution[0, 1], 100];
data2 = RandomVariate[NormalDistribution[0, 1], 100];
data3 = RandomVariate[NormalDistribution[2, 1], 100];BoxWhiskerChart[{data1, data2, data3}]The means of the first two populations are similar:
TTest[{data1, data2}, Automatic, "TestDataTable"]The mean of the third population is different from the first:
TTest[{data1, data3}, Automatic, "TestDataTable"]The "third series" of measurements of the passage time of light was recorded by Newcomb in 1882. The given values divided by 1000 plus 24 give the time in millionths of a second for light to traverse a known distance. The true value is now considered to be 33.02:
lspeed = ExampleData[{"Statistics", "NewcombLightSpeed"}];SmoothHistogram[lspeed, PlotRange -> All]Use Chauvenet's criterion to identify outlying observations:
ChauvenetOutlier[d_, data_] := Probability[x <= d, xNormalDistribution[Mean[data], StandardDeviation[data]]] < (1/2Length[data])outliers = Select[lspeed, ChauvenetOutlier[#, lspeed]&]newcomb = DeleteCases[lspeed, Alternatives@@outliers];A 
-test on the bulk of the data suggests that Newcomb's measure of the speed of light was significantly lower than reality:
Mean[newcomb]//NTTest[newcomb, 33.02, "TestDataTable"]The vitamin C content and head weight were recorded for 30 samples from each of two experimental cabbage cultivars:
ExampleData[{"Statistics", "Cabbages"}, "ColumnDescriptions"]cabbageData = ExampleData[{"Statistics", "Cabbages"}];cultivar = cabbageData[[All, 1]];Plots of the head weight and vitamin C content by cultivar:
w39 = Pick[cabbageData[[All, 3]], cultivar, "c39"];
w52 = Pick[cabbageData[[All, 3]], cultivar, "c52"];
c39 = Pick[cabbageData[[All, 4]], cultivar, "c39"];
c52 = Pick[cabbageData[[All, 4]], cultivar, "c52"];{SmoothHistogram[{w39, w52}, PlotLabel -> "Head Weight"], SmoothHistogram[{c39, c52}, PlotLabel -> "Vitamin C", PlotLegends -> {"c39", "c52"}]}The vitamin C content is significantly higher for the c52 cultivar:
TTest[{c52, c39}, 0, "TestDataTable"]The weight data is not normally distributed for c52, so MannWhitneyTest is used to show that a significantly lighter cabbage produced significantly more vitamin C:
DistributionFitTest[w52]MannWhitneyTest[{w52, w39}, 0, "TestDataTable"]Fifty samples from each of three species of iris flowers were collected. The samples consist of measures of the length and width of the irises' sepals and petals. It is difficult to distinguish the species virginica and versicolor from one another:
iris = ExampleData[{"Statistics", "FisherIris"}];species = iris[[All, -1]];versicolorData = Pick[iris[[All, 1 ;; -2]], species, "versicolor"];
virginicaData = Pick[iris[[All, 1 ;; -2]], species, "virginica"];A Hotelling 
test suggests a difference in the measures for the two similar species:
TTest[{versicolorData, virginicaData}, 0, "TestDataTable"]A visualization of the data suggests this difference is most prominent in the petal dimensions:
clmnDescr = ExampleData[{"Statistics", "FisherIris"}, "ColumnDescriptions"][[1 ;; -2]];
lgnd = Placed[LineLegend[{RGBColor[0.368417, 0.506779, 0.709798], RGBColor[0.880722, 0.611041, 0.142051]}, {"versicolor", "virginica"}, LegendLayout -> "Row"], Bottom];Legended[Table[SmoothHistogram[{versicolorData[[All, i]], virginicaData[[All, i]]}, PlotLabel -> clmnDescr[[i]]], {i, Length[clmnDescr]}], lgnd]Properties & Relations (11)
For univariate data, the test statistic follows StudentTDistribution under 
:
data = RandomVariate[NormalDistribution[], {1000, 15}];T = Table[TTest[i, Automatic, "TestStatistic", VerifyTestAssumptions -> None], {i, data}];DistributionFitTest[T, StudentTDistribution[14]]For multivariate data, the test statistic follows HotellingTSquareDistribution under 
:
data = RandomVariate[BinormalDistribution[.5], {1000, 25}];T = Table[TTest[i, Automatic, "TestStatistic", VerifyTestAssumptions -> None], {i, data}];DistributionFitTest[T, HotellingTSquareDistribution[2, 23]]The degrees of freedom are data-dependent for univariate data:
𝒹1 = RandomVariate[NormalDistribution[0, 1], 100];
𝒹2 = RandomVariate[NormalDistribution[0, 1], 95];
𝒹3 = RandomVariate[NormalDistribution[0, 5], 100];L[x_] := Length[x];v[x_] := Variance[x]TTest[𝒹1, 0, "DegreesOfFreedom"]L[𝒹1] - 1Two samples with equal variances:
TTest[{𝒹1, 𝒹2}, 0, "DegreesOfFreedom"]L[𝒹1] + L[𝒹2] - 2Two samples with unequal variances (Satterthwaite approximation):
TTest[{𝒹1, 𝒹3}, 0, "DegreesOfFreedom"](((v[𝒹1]/L[𝒹1]) + (v[𝒹3]/L[𝒹3]))^2/((v[𝒹1]/L[𝒹1]))^2 / (L[𝒹1] - 1) + ((v[𝒹3]/L[𝒹3]))^2 / (L[𝒹3] - 1))The type of degrees of freedom used can be controlled using VerifyTestAssumptions:
𝒹1 = RandomVariate[NormalDistribution[0, 1], 95];
𝒹2 = RandomVariate[NormalDistribution[0, 5], 100];Explicitly assume equal variances and test for normality:
TTest[{𝒹1, 𝒹2}, 0, "DegreesOfFreedom", VerifyTestAssumptions -> {"EqualVariance" -> True, "Normality"}]Explicitly assume unequal variances to use the Satterthwaite approximation:
TTest[{𝒹1, 𝒹2}, 0, "DegreesOfFreedom", VerifyTestAssumptions -> {"EqualVariance" -> False, "Normality"}]For multivariate data, the squared Mahalanobis distance is used to compute Hotelling's 
statistic:
data = RandomVariate[MultinormalDistribution[{1, 2, 3}, IdentityMatrix[3]], 100];MahalanobisDistanceSquared[data_, mu_] := With[{inv = Inverse@Covariance[data], m = Mean[data]}, (m - mu).inv.(m - mu)]Subscript[μ, 0] = {1, 2, 3};T2 = Length[data] * MahalanobisDistanceSquared[data, Subscript[μ, 0]]Under 
, the test statistic follows HotellingTSquareDistribution[p,n-1]:
pvalue = SurvivalFunction[HotellingTSquareDistribution[3, 99], T2]TTest[data, {1, 2, 3}, "TestDataTable"]If the population variance is known, the more powerful ZTest can be used:
σ = .25;data = RandomVariate[NormalDistribution[2.5, Sqrt[σ]], {1000, 15}];T = TTest[#, 2, VerifyTestAssumptions -> None]& /@ data;Z = ZTest[#, σ, 2, VerifyTestAssumptions -> None]& /@ data;α = 0.05;ZTest correctly rejects 
more frequently than TTest:
Probability[x < α, xT]//NProbability[x < α, xZ]//NTTest is robust to mild deviations from normality:
data = BlockRandom[SeedRandom[1];RandomVariate[𝒹 = StudentTDistribution[5], {1000, 35}]];Plot[{PDF[𝒹, x], PDF[NormalDistribution[], x]}, {x, -4, 4}, PlotLegends -> {"𝒹", NormalDistribution}]pvals = TTest[#, VerifyTestAssumptions -> None]& /@ data;The 
-value can still be interpreted in the usual way:
DistributionFitTest[pvals, UniformDistribution[]]Large deviations from normality require the use of median-based tests:
data = BlockRandom[SeedRandom[1];RandomVariate[𝒹 = CauchyDistribution[0, 1], {1000, 35}]];Plot[{PDF[𝒹, x], PDF[NormalDistribution[], x]}, {x, -4, 4}, PlotLegends -> {"𝒹", NormalDistribution}]tpvals = TTest[#, VerifyTestAssumptions -> None]& /@ data;srpvals = SignedRankTest /@ data//Quiet;The 
-value can be interpreted in the usual way for SignedRankTest but not TTest:
DistributionFitTest[tpvals, UniformDistribution[]]DistributionFitTest[srpvals, UniformDistribution[]]For two-sample testing of non-normal data, use MannWhitneyTest:
data1 = RandomVariate[LaplaceDistribution[0, 1], {1000, 35}];
data2 = RandomVariate[LaplaceDistribution[.5, 1], {1000, 25}];t = MapThread[TTest[{#1, #2}, VerifyTestAssumptions -> None]&, {data1, data2}];mw = MapThread[MannWhitneyTest[{#1, #2}]&, {data1, data2}];For non-normal data, MannWhitneyTest can be more powerful than TTest:
Probability[x ≤ .05, xt]//NProbability[x ≤ 0.05, xmw]//NTTest works with the values only when the input is a TimeSeries:
ts = TemporalData[TimeSeries, {{{1.224578634529677, 0.47929635789978015, 0.6572781300178168,
0.21496048742669355, 0.7299608014554928, -0.2495111111278263, -1.3286551762002712,
0.552725018274874, 0.19272112205837066, 1.1809144012420882, -1.1671 ... 40938613662046, 1.052394590214582, 0.9345044123980388, 0.38537803109557855,
-0.48660931166089394, -0.71203560340161}}, {{0, 100, 1}}, 1, {"Continuous", 1},
{"Discrete", 1}, 1, {ValueDimensions -> 1, ResamplingMethod -> None}}, False, 10.1];TTest[ts]TTest[ts["Values"]]TTest works with all the values together when the input is a TemporalData:
td = TemporalData[Automatic, {{{-0.25275046867718637, -0.7175779198306353, -1.9370139837317764,
0.006665621735740701, -0.3730807122324292, 0.6740106823161018, 0.8562214990564344,
0.955785083955732, 1.7020898014886303, 1.8523009430646802, 0.244 ... 3951759101545, -1.1611722313627828, 1.1602446901533021,
1.1052173095128992, 1.1089143920161917, -0.13837156328402}}, {{0, 100, 1}}, 2,
{"Continuous", 2}, {"Discrete", 1}, 1, {ValueDimensions -> 1, ResamplingMethod -> None}}, False,
10.1];TTest[td]data = td["ValueList"]//Flatten;
TTest[data]Test whether the means of the two paths are equal:
{data1, data2} = td["ValueList"];TTest[{data1, data2}]Possible Issues (2)
TTest assumes that the data is normally distributed:
data = RandomVariate[ParetoDistribution[1, 2], 100];TTest[data]
Use a median-based test that does not assume normality:
SignTest[data]The covariance matrix of multivariate data may not be invertible:
data = BlockRandom[SeedRandom[3];RandomVariate[SuzukiDistribution[0.3, 38], {15, 2}]];Quiet[TTest[data], {TTest::nortst}]
Neat Examples (1)
Compute the statistic when the null hypothesis 
is true:
data = RandomVariate[NormalDistribution[], {2500, 100}];T1 = TTest[#, 0, "TestStatistic", VerifyTestAssumptions -> None]& /@ data;The test statistic given a particular alternative:
T2 = TTest[#, 1, "TestStatistic", VerifyTestAssumptions -> None]& /@ data;Compare the distributions of the test statistics:
SmoothHistogram[{T1, T2}, Filling -> Axis, PlotLegends -> {"SubscriptBox[H, 0] is True", "SubscriptBox[H, 0] is False"}, PlotStyle -> Thick]Text
Wolfram Research (2010), TTest, Wolfram Language function, https://reference.wolfram.com/language/ref/TTest.html.
CMS
Wolfram Language. 2010. "TTest." Wolfram Language & System Documentation Center. Wolfram Research. https://reference.wolfram.com/language/ref/TTest.html.
APA
Wolfram Language. (2010). TTest. Wolfram Language & System Documentation Center. Retrieved from https://reference.wolfram.com/language/ref/TTest.html