Calculating Statistical Values on Different-Sized Subsets of Data

Chris has a huge amount of data in a worksheet, and he wants to analyze the data based on different groupings within it. For instance, he has data in cells A2:B36001, where row 1 contains the column headings Time and Signal. He wants to divide the data into groups consisting of some arbitrary number of sequential values, and then extract, for each group, a mean value for the Time, a mean value for the Signal, and a standard deviation for the Signal.

The easiest way to handle this type of requirement is to add a column that is used to indicate a group number for each row. Follow these steps:

1. Put the heading Group into cell C1.
2. In cell E1 enter the number of values that should be in each group. For instance, if you want each group to contain 10 sequential values, enter the number 10 in cell D1.
3. In cell C2 enter this formula: =INT((ROW()-ROW($C$2))/$E$1)+1
4. Copy the formula in cell C2 to the range C3:C36001. Column C now contains a "group number" for each row, based on the value in cell E1. If E1 is 10, you end up with 3600 groups, 1 through 3600. If E1 is 100, you end up with 360 groups, 1 through 360.

With the group numbers set up, you are ready to do the analysis. There are a couple of ways you can do this. One way is to use the subtotaling capabilities of Excel. Select one of the cells in the data area and follow these steps:

1. Choose Subtotals from the Data menu. Excel displays the Subtotals dialog box. (Click here to see a related figure.)
2. Change the At Every Change In drop-down list to Group.
3. Change the Use Function drop-down list to indicate the type of statistic you want to calculate for each group.
4. Change the Add Subtotal To area so that only Time or Signal are selected, as appropriate.
5. Click OK.

Excel groups and subtotals the data, as directed. You can hide the detail (and only show the subtotals) by clicking on the small 2 (with the box around it) in the outline area at the left of the worksheet. If you later want to change what is calculated, or if you need to change the number of items in each group, just remove the subtotals (Data | Subtotals | Remove All) and repeat the above steps.

Another way to derive the statistics from your data is to use a PivotTable. Make sure that there are no subtotals in the data and select a cell within the data. Then follow these steps:

1. Choose PivotTable and PivotChart Report from the Data menu. Excel starts the PivotTable and PivotChart Wizard.
2. Click Next. (The default selections in step 1 are OK.) Step 2 of the PivotTable and PivotChart Wizard is displayed.
3. The entire range of your data (A1:C36001) should be selected. Click Next. Step 3 of the PivotTable and PivotChart Wizard is displayed.
4. Make sure New Worksheet is selected, then click Finish. Excel creates a blank PivotTable and displays a field list.
5. Drag the Group field to the Row area.
6. Drag the Time field to the Data area.
7. Drag the Signal field to the Data area.
8. Drag the Signal field, once again, to the Data area. The PivotTable should now show "Count of Time," "Sum of Signal," and "Sum of Signal2" for each group.
9. In the Data area, right-click one of the "Count of Time" labels. Excel displays a Context menu.
10. Choose Field Settings from the Context menu. Excel displays the PivotTable Field dialog box. (Click here to see a related figure.)
11. In the Summarize list, choose Average.
12. Click OK. All the "Count of Time" labels change to "Average of Time."
13. In the Data area, right-click one of the "Sum of Signal" labels. Excel displays a Context menu.
14. Choose Field Settings from the Context menu. Excel again displays the PivotTable Field dialog box.
15. In the Summarize list, choose Average.
16. Click OK. All the "Sum of Signal" labels change to "Average of Signal."
17. In the Data area, right-click one of the "Sum of Signal2" labels. Excel displays a Context menu.
18. Choose Field Settings from the Context menu. Excel again displays the PivotTable Field dialog box.
19. In the Summarize list, choose StdDev.
20. Click OK. All the "Sum of Signal2" labels change to "StdDev of Signal2."

You now have the data desired. You may prefer a setup that shows only one row per category for your data. If so, simply click on the Data header and hold down the mouse button as you slowly move the mouse to the right. As you move the mouse pointer into the Total header area, a small icon near the mouse pointer shows a "change" in the layout. Release the mouse button and you will have only one row for each group in your data. (Click here to see a related figure.)

If you need to format the columns (such as changing the format for the Average of Time column) you can do so. If you need to change the number of data items in each group, just go back to the data worksheet and change cell E1 to a different value. You can then return to the PivotTable, right click it, and choose Refresh Data.