11 edit scoring function
Edit Scoring Function
Even though the A-B-C prioritization scheme is helpful in establishing the relative severity of the various edit exception codes, there are still many exceptions gathered under the “A” list umbrella. This still complicates the process of identifying the conditions that cause the most trouble to macro data and the EQUI deliverable. A typographic error with an extra digit in one of the months of employment (4000 versus 400 employees, for instance) may not be easily spotted among an edit report of considerable size, or in a brief perusal with the ES2E screen. A more precise targeting mechanism could be helpful in accelerating the resolution of the most significant data entry problems.
To address this problem, the Edit Scoring function was devised. Although this has not been finalized - and will continue to be augmented for improved targeting efficiency - this function appears to be effective in identifying the worst-of-the-worst accounts in regard to significant discontinuity of employment and/or wage data. This method was first included in Version 6.0’s edit report output and the ES2E screen display. It may also be included as a sort parameter (with a letter “F”) in edit report sequencing. One of its primary uses currently, however, is in the FESTER (Focused Error Scoring Targeted Editing Report) processing of Job 008D. This job can give added or reduced emphasis to particular error codes, and includes a score cutoff parameter to ignore the less significant error conditions.
The score value is designed to give a numeric value that interprets how questionable the quarter’s data appear to be. The higher the value, the more suspect the data seem to be. The following scale serves as a rough measure as to how the scores should be interpreted:
| Score Values | Interpretation |
|---|---|
| 20 and higher | Extreme Errors |
| 15 to 19.999 | Severe Errors |
| 10 to 14.999 | Significant Errors |
| 5 to 9.999 | Typical Conditions |
| 0 to 4.999 | Isignificant |
| < 0 | Disregard Fully |
The possible range of a score value is approximately 8 to +70. Most records with edit exceptions will be in the range of +3 to +8. The default cutoff for FESTER is +10, which will generally produce a report that will generate one page of exceptions for every 15,000 establishments in the State. Yet, this will vary significantly depending on the phase of the editing cycle. The beginning of an edit will show a larger report, while the final clean-up could be a near-empty report.
The scoring function begins with a “Base Score,” determined from the employment and total wage values of the edited quarter and the immediate prior quarter. This base-score value is later augmented by the most severe edit condition present for the account (according to the A-B-C hierarchy).
The base score has two components (employment-based and wage-based), which are summed together. They form the equation shown below:
Here, the components can be clearly seen, with the employment and wage portions each delineated by a pair of square brackets. The first uses three employment fields, namely the maximum (Max), large and small values. The maximum employment is the largest of any of the three months of employment in the edited quarter, or the third month of the previous quarter. The large and small values are the two adjacent months of employment involved in the largest employment difference found through the quarter. The small and large wage fields refer to the edited quarter and previous quarter total wage values. They are assigned to the fields shown here depending on which is larger. The superscripts of .25 and .1 are equivalent to the 4th root and 10th root of the values shown, respectively. By summing the two components, the base score can be calculated.
In Version 6.2, a transition of the scoring formulation began. The formulation shown above is still used in the batch an on-line edit scoring, but will become obsolete by Version 6.3. The new FESTER scoring, employed by the 008D job, is shown below as V6.2:
The revised formulation uses the same processing for the employment component, but has the following adjustments to the wage component. First, it uses the eighth root (.125 power) instead of the tenth root (.1 power). It scales this root to a 0-to-1 wage ratio range (Wage Diff / Wage Large) instead of a weighted ratio (using half the smaller wage) that gives a 0.5-to-1 ratio range. This method also does not halve the first result, and adds in half of the eighth root of the difference as a final weighting factor. These changes generally produce a larger value in the wage component, but one that provides a truer weighting of total wage fluctuations.
The final form that will be introduced with Version 6.3 will scale both sides down slightly. The reason for this is that an employment change from 0 to 1 and a total wage change from $0 to $1 can grant a fairly hefty score (about 2 points) when essentially nothing has happened. So a minimal change will be scaled to a zero weighting. The V6.3 format then becomes:
An equivalent form is to say that the V6.3 score is 2.5 points lower than the V6.2 score, due to two decrements of one point, and the removal of half a point at the end of the equation. So an equation could be written as:
As a sample of how the calculation is enabled, we will use an account with four months of employment (June through September of 2001), and two quarters of total wages (second and third quarter of 2001), as follows. Employment values are 10 (June), 19 (July), 20 (August), and 12 (September). Total wages values are $121,450 (2nd quarter) and $177,222 (3rd quarter). From these, EmplMax = 20 (the largest value found), EmplSmall = 10, and EmplLarge = 19 (since the 9-employee jump from June to July is the largest monthly value change). Of course, WageLarge = 177222 and WageSmall = 121450.
From these values, the V6.1 base score can be computed from:
Under the “V6.2” formulation, the score would be 6.502021, and the “V6.3” formulation gives a score of 4.002021. Looking at the above data, there is little that could be considered alarming, with a largest employment difference of 9 and an AQW change of only $300 (considering the increase in employment that accompanied the wage increase). So a score that falls below 5 would seem appropriate. The V6.2 method is probably a little on the high side, which gives good reasoning for anticipating the final V6.3 format.
| Code | Adjustment |
|---|---|
| A | +7.5 |
| B | +3.5 |
| C | -0.5 |
| D | -4.5 |
| E | -8.5 |
The precept behind the score base is that accounts with higher levels of employment or wages and higher levels of fluctuation in those fields are the ones most likely to have a data problem. Simultaneously, even if the account doesn’t have a particular employment/wage error flagged, a higher score-level would force any other edit exceptions closer to the top of the pile. The fourth root and 10th root (to be 8th root) computation is necessary, since there are such wide ranges of these two field, especially on the wage side. A change in total wages from $10 to $10,000 seems immense in percentage terms (a 1000-fold increase), but is miniscule when examined against the billions of dollars in the statewide wage data. So, the .25 and .1 (or .125) exponentiation serves as a scaling factor to examine the fields in a relatively manageable framework. On the employment side, the largest employment difference acts as an additional scalar, to adjust the employment magnitude by the volatility of the data, multiplying by a value from 1 to 2. The wage fluctuation adjusts the wage magnitude with a multiplication by some value between 0.5 and 1. Wages inherently have higher volatility than employment, so must take a back seat in the score calculation.
Adjustments for the base score examine the A-B-C classification of each associated micro edit exception, and slide the final score accordingly. The final adjustment is the diminution of the score by the error number (since the larger the error code number, the less significant the edit exception is). Since the identical employment edit (code 128) is regarded as less significant than other C-level error codes, it receives a “D” classification. The missing EIN condition (code 116) is even less-regarded, so is granted a level of “E”. Score adjustments by level appear in the following table.
With this adjustment, the final score value for an account is as follows:
In which AdjABC is the adjustment from the table to the left, and ErrCode is the error code numeric value. In the sample shown earlier, if the most significant error had been 024 (reactivation date format or sequence), the final score would be calculated from:
Where the score function really comes in handy is in detecting probable typographic errors. For instance, if the data entry of an account with 13 employees was keyed without the appropriate tab characters, so that the employment appeared as 131313 in the first month, and zero in the last two months, a high score would be produced. If we were to have a prior quarter third month employment of 12, and total wages changing from $142,550 to $147,388, then the account (with a most severe error of “A091”), the score would be calculated as:
Compared to a V6.2 score of 44.77651 and a V6.3 score of 42.27651, all of which indicate a high degree of certainty that the data are severely out of line with what could be consi¬dered reasonable. Such targeting of the most likely severe errors can be of great assis¬tance when delving through the mass of accounts listed in the edit listings. The A-B-C hier¬archy, with a complete list of error codes, appears next.
If the employment were to be corrected in the above example, so that the three ‘13’ values were separated into their respective employment months, the equation would be completely different. Since the employment change error would no longer be flagged, the score would need to refer to the next most significant error. In this case, let’s assume that there was also a problem with the physical address format (such as the presence of a post office box number rather than a street address). This generates an error code of 114, which carries a level “C” ranking of importance. The revised V6.1 score then becomes:
A sub-zero score, denoting an account that should be ignored, as far as employment and wage errors are concerned.
Related Links
