Degrees of freedom kind of measures your computed mean's ability to move about, and it is #dataPoints - 1 ( v in the link I provided). The more datapoints you have, the smaller the interval around mean probably is. This is related to the number of datapoints you have. Now, before you click that, you need to know about 2 more things: Degrees of Freedom You could test the other way around by saying:Īfter you computed these t-values, you can tell whether they're significant by looking at the critical value table. The example t value I showed tests whether the mean timing results for core count of 1 is significantly different than the timing results for core count of 2. Compute a t-value: t = (mean_1 - mean_2)/(s_1/ sqrt(#dataPoints)) Now you will have a mean and standard deviations for each core count: (m_1, s_1), (m_2, s_2), etc. The standard deviation should be the population standard deviation, which is the square root of population variance Population variance is (1/N) * summation_for_all_data_points((datapoint_i - mean)^2)
The student's t-test essentially says "you calculated an average time for this core count, but that's not the true average. If your job takes a long time, reduce the # of trials, all the way down to 5 or so, but remember that more trials will give you a higher degree of confidence. Run timing results for ~32 trials for each core count and get a confidence interval, then you can say for certain whether running on all cores is right for you. EDIT: It looks like you're only using 4 cores, so time with 1, 2, and 4. I don't know the size of your job, but you should probably run some scaling experiments where you time your job on 1 processor, 2 processors, 4 processors, 8 processors, until you hit the max core count for your system (typically, you always double the processor count). For small batch jobs, running on a single core or two cores is much faster due to the fact that you're not paying that overhead.
Any time you do parallel processing there is some overhead (which can be nontrivial, especially with locking data structures and blocking calls).