Statistical Analysis of ATA's Survey

Cristina Márquez de Camihort

This is a statistical analysis of the Survey used by the ATA to justify its decision to offer accreditation exams abroad.

ATA’s report states that the Survey was conducted during the month of February. In reality, the survey was e-mailed on February 14, 2000; the deadline to reply was February 23, 2000. That is 9 days, not one month.

We don’t know how many members received the message or the form, either by fax or by postal services. The message from ATA stated that they were not sure who was actually getting the survey.

This survey is being provided by e-mail to those members for whom we have an address; by fax to those members for whom we have no e-mail; and by postal service to other members. We are trying our best to reach all members within the time frame provided. If you know of a member who did not receive this form, please forward a copy for their use.

Margin of error

As stated by the report:

All told, e-mails or forms were sent to 7,433 members (6,299 via e-mail notification, 584 via regular mail and 550 via fax). A total of 1,875 completed forms were received in time for processing, representing a 25% response rate. Based on 1,875 presumably random responses, this study has a +/- 2% margin of error, at a 95% level of confidence. (In other words, we can be 95% certain that for a question with 1,875 responses, the survey’s figure would be within +/- 2% of the actual universe’s response.)

The +/- 2% margin of error on a 50/50 opinion is a deviation value high enough to invalidate the results.

After calculations, the results indicate that the 49.8 figure in the report for members opposing the sittings abroad is not accurate: the exact figure is 49.89 while the 50.2 in favor is actually 50.10.

According to the report, the percentages are based on 1,875 presumably random responses. I wonder what "presumably" means here.

The members residing outside the US were also part of the survey. Based on their percentage (18.25%) and even when they are a clear minority, their incidence on the final results due to their level of participation in the survey, is high enough to polarize them. A "normal distribution of data" means that most of the examples in a set of data are close to the "average", while relatively few examples tend to one extreme or the other. In this case, foreign based members have more than doubled the amount of US based members who participated in the poll.

The participation is as follows (count done on the 1999 Membership Directory):

Foreign based translators 642

US based translators 6791

Foreign based opinions 335

US based opinions 1519

% of foreign opinions 52.18%

% of US opinions 22.37%

National or International ATA?

The results indicate that 81% of the participating members leaned towards a National ATA: 57% of them to a National Association with an international orientation, and 24% of them to a National Organization. In spite of that, the Board voted that they want an International organization, with exams offered abroad.

Is it a coincidence that the percentage of members whose opinion showed a preference for an International organization (19%) is equal to the percentage of foreign members who participated in the survey (18.2%)?

The decision to lift the suspension on accreditation examinations outside de United States is also questionable based on the Survey's following statement.

Just 53% of the respondents who gave an opinion thought it was likely that making accreditation examinations available to members living outside the U.S. will create a group of translators who will be able to undercut unfairly the rates charged by U.S.-resident translators, while 37% predicted that this turn of events was not likely.

Just 53% means that the majority agreed on this point.

There is another interesting result in the section titled "Mean Degree of Agreement with Various Statements":

Regardless of economic impact, the ATA should not offer examination sittings outside the U.S. if the majority of members are upset by this policy (67.5% agreed)

In a 1 to 5 scale, where a higher value means a stronger agreement, the results show a 3.66 value in favor of this statement (73.2%).

Considering the biased percentage of foreign based opinions, the +/- 2% margin of error and the 49.89% opinions against the sittings abroad versus 50.10% in favor of them, it's difficult to understand how the Board interpreted that the majority of members were not upset by this policy. This 73.2% (or 67.5% as showed by the survey), clearly indicates that the membership was not "placing the burden of a decision squarely on the Board", as stated by the Board of Directors memo sent on March 13, 2000.

Biographical Note

Cristina Márquez de Camihort is a Technical-Scientific translator based in New York, specialized in SW Localization. She holds a Computer Science degree and has worked in the Translation field since 1979, when appointed to the position of TS Coordinator at the IBM Latin American Translation Center, Buenos Aires, Argentina.