What do you do to determine whether some experimental variable had any effect on your data? You compute an ANOVA, of course. But what do you do, when you only got one subject (sorry, participant)? Well, you could compute a randomization test.

How it works

The little program I wrote, will perform a randomization test according to Edgington (1992). Suppose you've got 10 measurements, or the means or medians of 10 measurements. 5 of these resulted from condition A, and the other five from condition B. RNDTST will take these 10 data and compute the mean of each condition. The difference of those means is the critical value. RNDTST will now reassign the data to the two conditions in every possible way, compile the means of the new pseudo-conditions and compare their difference with the critical value. If the difference is larger than the critical value (or smaller if the critical value is negative) the hit count will be increased by one. If the data were not influenced by the experimental treatment (i.e. there is no effect of condition), the hit count should be quite large. If, on the other hand, there is an influence of the treatment, the chance assignments of the data to the two conditions, should produce only very few hits.

The number of all possible assignments of N data to two equally large groups is N over N/2, or N! / ((N/2)!)2. In the case of 10 measurements this would give us 252 possible assignments (combinations). With an alpha of .05 we should hope for less than 12 hits. Otherwise, we would have to conclude, that the difference of the means of our two conditions might have been pure chance.

Running RNDTST

Drop to the DOS prompt (or to whatever shell you use on your system) and enter

RNDTST file-name
where file-name is the name of the file that contains your data. The data should be arranged in rows. One row, one number. The first half of the rows/data should belong to your first condition, the second half to your second condition. If you want a two-sided test, add the number 2 after the file name.

Download

You can either download the source code (1,998 bytes) or a precompiled binary (83305 bytes) for DOS/WIN. The sources should compile with any C++ compiler. Request optimizations when compiling, though.

Limitations

The total number of data is limited, because the possible number of combinations soon leaves the range of valid values for a C int. To be honest, I don't know the exact maximum number. But I guess it should work with up to 40 data. If you happen to find any bugs, please send me an email. If RNDTST breaks your computer, does harm to your sanity, eats your lunch, or hides your keys, please don't blame me. I refuse to take any responsibility.

The only reference

Edgington, E. S. (1992). Nonparametric tests for single-case experiments. In T. R. Kratochwill & J. R. Levin (Eds.), Single-case research design and analysis (pp. 133-157). Hillsdale, NJ: Lawrence Erlbaum.