The Uses and Abuses of Surveys in Service Development Planning for Disabled People: the Case of Lesotho

Chris Saunders & Susie Miles, Save the Children-UK March 1990

In our experience ministries are very keen to have base-line data which they believe will help in the planning of services for disabled people. As a result, they often request outside agencies to conduct and/or finance disability surveys. In this paper we will argue that:

  1. There is sufficient relevant data for planning purposes;
  2. Data should be derived from existing services;
  3. The methodology used is often suspect;
  4. Surveys can be wasteful or even counter-productive;
  5. Service development should take priority.

PLANNING

There is enough information currently available to demonstrate that the common conditions which result in impairment and disability occur at a relatively predictable rate, given a knowledge of the particular circumstances of the country. The single most important question relates to the effectiveness of the immunization programme. Where polio is still endemic, the overall prevalence of physical impairment is higher than in countries where new cases of polio are rare. The incidence of polio will be an influence on the style of service provision necessary because the needs of a person disabled by the effects of polio are usually very different to those of a person disabled by cerebral palsy.

With the information on polio and knowledge of survey results from other countries, it will be possible to predict the prevalence of impairment and disability in Lesotho. There are, however, circumstances special to particular countries which will affect the overall prevalence rate of impairments and disabilities in a country. In the case of Lesotho the most influential will be the effect of mining injuries on the adult male population. Additional factors also need to be considered, for instance whether the mountain population is also affected by iodine deficiency. This would need a separate investigation, and one has been completed recently (Todd, 1988). Conflict will have a major impact on the prevalence of disability in many countries of the world.

Compiling data from the above will give sufficiently accurate information in the planning of services in Lesotho. The main problem will be to establish commitment to the development of comprehensive services for disabled people, and this is a qualitative, rather than a quantitative issue.

Having established this commitment, it is then crucial that these services develop in the most appropriate way possible. We would argue that this should be a community-based approach and establishing this point comes from a true understanding of disability issues, rather than a knowledge of statistics related to prevalence.

METHODOLOGY

We are not suggesting that data is not important, but it is the way data is collected, and then used, which is at issue.

A typical approach would be to conduct a sample survey which would be based on interviews with several thousand heads of households. The interviewers would ask questions about the health of people living in the household, and if there were doubts about this, trying to establish whether the child or adult was disabled, or not. The impairments would then be classified, and the prevalence of certain conditions calculated.

Such surveys often derive a spurious credibility from the methodological sophistication that can be applied to sampling and statistical analysis techniques. But the credibility is unjustified, given the crudeness by which the impairments and disabilities are defined and measured.

Disability can only be defined by context. A classification system based on medical diagnosis tells us little about the person's real problems. For instance, the terms cerebral palsy or post-paralytic polio will give little clue as to whether the children or adults they are applied to are able to walk, integrate with family, peers, attend school and so on. It is these factors which are most relevant to rehabilitation and yet are most difficult to measure and categorise statistically.

The disability that results from physical impairment is very much dependent upon the physical environment and social context. The child who has mobility problems and lives in the mountains may be extremely disadvantaged both in terms of his/her ability to get to school and his/her potential for earning a living. The same child living in a town with tarmac roads and a school nearby may have few disadvantages compared with his/her peers.

Even if disability can be measured in some quantifiable way, it is not necessarily possible to determine its effect on the individual. Hearing-impairment is a good example of this. Two individuals may have an identical hearing loss, according to their audiogram (a graph which shows the level and type of loss), but, because of different factors, such as time of onset of the problem, parental support, linguistic ability, availability of a hearing aid and so on, one individual may be far more disabled than the other.

The same applies to learning disability. It can be argued that this condition can in some way be measured through IQ or attainment tests, although the validity of their application in countries like Lesotho is questionable. Even if some form of measurement can be obtained, it offers no indication of the child's social and functional abilities.

Some impairments are accepted as part of the ageing process and are totally acceptable, but these same conditions in a younger person would be considered a major problem. For instance, an elderly parent whose mobility, vision and hearing are becoming increasingly impaired may still remain a respected member of the household. A child with similar problems may receive little respect, and may even be neglected as a result of these impairments.

All of the above points lead us to conclude that the enumeration of impairments and disabilities is not a helpful way of defining service needs in a country.

Clearly certain types of service need to exist. For instance, we know that there must be facilities for treating conditions such as otitis media. There is sufficient information from other surveys to tell us that this is a very common problem in any country. It is a condition, which, if left untreated, will lead to hearing-impairment in a proportion of those suffering from it. Whether it has a prevalence of 25/1000 or 125/1000 in the child population of Lesotho is of little relevance to the question of whether treatment needs to be available or not. Treatment should always be available. The condition is a common problem that can lead to a significant impairment and disability, if left untreated, or treated inappropriately.

In the same way, we know that where poliomyelitis is present there will be a need for an orthotics service, and this service will be necessary for many years after the eradication of polio. Without such a service, the quality of life of many who contracted paralytic polio will be significantly reduced. There is no question that such a service should exist, only a question of how best to make such a service available to all who need it. A sample survey cannot answer such a question.

We know that if disabled children are going to be given the same opportunities as their able-bodied peers that special educational facilities need to be established in some form or other. The questions we need to answer are not whether such a service ought to be established, but what sort of service it ought to be - should it be based on specialist schools, resource units, integration into mainstream, or some combination of all three? This question can't be answered through a survey, but only through the careful evaluation of existing programmes in the region, consideration of population distribution factors, and resources available to establish the service.

It is as valid that demand determines service development, as statistical data acquired through a survey.

Once basic facilities have been established, then information should be collected to guide the development of these services. Information collected though a service context, by the practitioners who are already working with disabled people and their families, is likely to be of a higher quality. Their service role means that they are in a position to obtain information that is impossible to collect through an enumeration survey method.

It is really only practitioners who can define the qualitative needs of individuals. It is only they who are able to determine the skills and facilities necessary to meet these needs. Consequently we would argue that information collected through surveys will be of little value compared with that collected through the service delivery context.

The Kibwezi rural CBR project in Kenya has been running since 1982. This programme has been steadily expanding, and it was argued that as part of this process, a comprehensive survey of disabilities should be conducted. This took place in 1987, and included the surveying of parts of the existing area covered by the CBR programme, and a new area. Part of the rationale for this was that there was considerable knowledge of families already involved in the programme, but it was necessary to find out whether other families within the same area had been missed out. In addition, it was thought timely to expand the service into a new area, and to identify new families for the programme in as systematic way as possible. A house-to-house survey was conducted using local Community Health Workers (CHWs)

Reference is made to the results of this study later in the paper. However one relevant finding relates to the type of interventions required in order to meet the particular needs of children and families newly-identified by the survey. The existing CBR programme was able to do this. If there had been no programme a wide range of problems would have been identified, recorded and then ignored indefinitely. Such a situation cannot be justified.

Table 1 is adapted from the survey report (Baldwin et al, 1989), and illustrates clearly the range of interventions necessary for an effective response to disabilities occurring in a community survey. It is important to note that some children were referred to more than one intervention. For instance, home-based follow-up would be recommended for most, if not all, children referred for surgical intervention, special schooling and so on.

Tabel 1

Intervention or activity No of new children referred No of children attending
Playgroups 7 6
Placement in Special school 11 2
Surgery 10 9
Epilepsy clinic 10 10
Mobile eye clinic 8 5
ENT clinic 15 4
Placement in ordinary school 26 1
Continuing in ordinary school 41 38
Home-based programme 14 14
Speech therapy 3 0
Applied nutrition 1 1

EFFECTS

One of the most negative effects of a survey can be that of raising expectations which cannot be fulfilled by service delivery methods. This often comes about because governments argue that they need data on prevalence of disability before they will commit themselves to service development. Consequently the effect of a survey is that the provision of services tend to be delayed, and therefore there is inadequate means of meeting expectations.

It is often argued that a disability survey has the positive effect of raising public awareness. However we would argue that, if this is the goal, there are far more effective and economical ways of raising public awareness of disability than organizing a survey.

SURVEYS AND RESULTS

Large amounts of information are contained within current data bases. The most recent and comprehensive example of this is the WHO compendium on disability surveys worldwide (WHO, 1988).

In reviewing survey results, there are often dramatic discrepancies in prevalence rates for the same conditions between countries. These discrepancies may often be ascribed to 'real' differences in the numbers of people with those disabilities. However we would argue that the discrepancies are probably far more a function of a difference in definition, than in numbers. Therefore one can arrive at the situation of a child survey in Nigeria presenting the prevalence of severe visual disability and blindness as 0.5/1000 of the child population: this being the smallest major category of disability in the survey results (Saunders, 1984). Compare this with the Zimbabwe survey of 1982 (Davies, 1982), where visual disability was the largest single category of disability in that survey. Take the same two surveys: in the Nigeria study, hearing-impairment was the largest single category, whereas in the Zimbabwe survey, it is not mentioned as a discrete category.

We are certain that neither researcher would argue that such a discrepancy reflected massive differences in terms of definition, categorisation and sampling procedures. Neither person is right or wrong in this, rather decisions were made to record, classify and interpret results differently.

The above points go to emphasise the problem of interpreting the results of blanket surveys. Often authors of the final report resort to making the same conclusions as could be drawn by informed workers who know the country or region and are aware of the main issues relating to disability. The conclusions are rarely directly related to the data obtained from the survey, but are usually some generalised conclusions concerning the fact that the majority of the disabled children identified in the survey receive little or no support or treatment, that they are disadvantaged in some way compared with their peers, and so on.

The Kibwezi team write the following in their concluding statements: "It must be emphasized at the outset that this survey was not undertaken for the single objective of estimating the prevalence and causes of disability …… It was designed to involve the community and those representatives of the community, CHWs, in identifying disabled children and in learning what might be done by both health services and the community to help them achieve, as far as possible, their full potential …… The pattern of disability did not differ markedly from the results of other surveys and was predictable. The 6% prevalence rate for disability has been found widely in childhood surveys and is probably at the lowest limit of the true disability rate."

There are therefore some firm conclusions that can be drawn from existing survey information.

HEARING-IMPAIRMENT

Congenital deafness has a fairly uniform incidence of around 1/1000. In developing countries acquired deafness at an early age would contribute another 2/1000 due to conditions such as meningitis. We can therefore assume that the total incidence of deafness in the child population in Lesotho will be in the region of 3/1000 (Newell et al, 1987)

Children with moderate to severe loss, who can benefit from ordinary education with the help of a hearing aid, will be in the region of 4/1000 (Newell et al, 1987).

Otitis media is an extremely common condition in all countries. Estimates of its incidence are often as high as 20% in the child population. The number of children who suffer hearing loss as a direct result of this condition is unclear. It is, however, well known that there is a strong association between otitis media and hearing loss, and this is often at a significant level.

Children with mild hearing losses make up the largest group of hearing-impaired children. It is within this group that the biggest discrepancies occur in the survey data. This is as much a function of defining the significance of hearing loss, as the sampling and measuring procedures. Consequently one could find results suggesting that as many as 17% of all children have some form of significant impairment. Such a statement can be justified statistically, but the problem of defining significant becomes a major issue. In the European context, the slightest impairment is of significance. In the Lesotho context, this group would not warrant attention.

VISUAL-IMPAIRMENT

Although visual-impairment is represented as a prominent disability in some surveys, in the child population the number with severe problems is relatively small. Usually this is estimated as being less than 1/1000 (Baldwin et al, 1989, Saunders, 1984, 1985).

Eyesight tends to deteriorate with age and many elderly people have difficulty seeing. In many cases this is remediable with glasses.

Defining visual-impairment is difficult because cut-off points cannot easily be determined. The question of glasses also complicates this since correction is a relatively simple process in most cases.

LEARNING DISABILITY

Down's Syndrome is the single most important condition leading to severe learning disability. Down's Syndrome has a relatively uniform incidence rate across populations of 1.6/1000 (Baldwin et al, Saunders, 1984, 1985). Survival rates (ie the prevalence) in countries differ according to available medical facilities in a country. In rural areas of Lesotho the chances of a child with Down's Syndrome reaching late adolescence are slight. With reasonable medical facilities most children with Down's Syndrome will have a good life expectancy.

It can be assumed that 2/1000 children will have conditions that will lead to severe learning disability, often in conjunction with other disabilities. These may result from a number of rare genetic conditions, or diseases such as meningitis. As with Down's Syndrome, the survival of these children depends upon the availability of medical services and the probability of such children reaching adulthood is relatively slight (Baldwin et al, Saunders, 1984, 1985).

The conditions that lead to mild to moderate learning disability are not usually medical. They are more likely to be a result of a mixture of poor nutrition, under-stimulation and so on. Within the population, the prevalence of such a disability can be estimated statistically using measures of intelligence. This, of course, has little functional value. In surveys prevalence estimates vary widely depending upon the method of measurement employed. By definition an IQ screening method will lead to relatively high numbers of children being identified. If parental anxiety over educational progress is used as a means of identifying learning disability, this will lead to variable results, depending on the value placed on the educational system. Measures of social and functional attainment will probably result in relatively low numbers because, in fact, children with mild learning disabilities are usually able to function very well in family life, and are therefore not noticeably different from their peers (Baldwin et al, Saunders, 1984, 1985).

CEREBRAL PALSY

Cerebral palsy is a complex condition as far as prevalence is concerned. It is certainly the major physical impairment which affects children in developed countries. Its occurrence is associated with a number of other factors, including low birth weight, poor birth practice and early childhood diseases such as meningitis. The two main points about the incidence of cerebral palsy are that:

  1. In developing countries the incidence rates are extremely high, but few children survive. In developed countries incidence rates are relatively low, but most of these children survive because of good medical care.
  2. As medical services in developing countries improve, the chances of survival for children with cerebral palsy also improve. It is noticeable that as the incidence of polio decreases, with improved vaccination coverage and health education, cerebral palsy becomes a more prominent condition.

The prevalence of cerebral palsy in the population is difficult to measure for a number of reasons. However, in a country like Lesotho, it would be safe to assume that it would have a prevalence rate of about 2/1000 (Baldwin et al, Saunders, 1984, 1985).

POLIO

Of all the disabling conditions, polio has been the most researched, in terms of prevalence data in developing countries. Estimates of prevalence range from 2-10/1000 in a non-immunised population (Bernier, 1983). Vaccination coverage will obviously affect this rate significantly. The most recent survey in Lesotho suggests a prevalence of around 2.2/1000 in the child population (Ward, 1981). This rate includes all children showing evidence of paralytic polio. The prevalence of children who have severe mobility problems as a result of this is smaller and much depends on definition of disability. However we can assume that approximately 60% of this number will have relatively minimal mobility problems, the remaining 40% will require some form of intervention. If this is provided appropriately, severe disability can be prevented in the majority of such children. These figures are extremely conservative given that the data being used is nearly ten years old. EPI coverage has improved since then and the number of new cases presenting to hospitals and clinics should now be relatively small.

OTHER PHYSICAL IMPAIRMENTS

Spina bifida and hydrocephalus in conjunction with a number of less common conditions will contribute a small proportion of the total demand for services for physically disabled people. It may be safe to assume that collectively this is less than 2.5/1000 in the child population, although studies suggest that figures will show some variation.

Club foot seems to be relatively common in many African countries. The Kibwezi survey, for instance, recorded a rate of 2.4/1000 (Baldwin et al. 1989). It is a condition, which, in most cases, is easily remediable during the first three months of life. The longer it is left, the more problematic it becomes for remediation. The exact incidence of the condition in Lesotho is of academic interest only. What is essential, is that health workers are alerted to the importance of early identification and referral, and that an effective service exists for treatment and follow-up.

Burn injuries are also likely to be relatively common in situations where open fires or kerosene stoves are used at ground level for cooking, and where young children move around freely in the cooking area. The authors have no specific data on the incidence of burns and their consequences. However education on prevention must be provided through a variety of channels, and an effective service must exist for treatment and follow-up.

Accidents and mining injuries represent a major category of conditions in Lesotho. If specific information is needed for planning purposes, this can be obtained through existing record systems, rather than through surveys. However a major service initiative is underway at this point in time in Lesotho, and clearly the addition of specific survey data is unnecessary.

EPILEPSY

Prevalence of this condition is extremely difficult to estimate. The prevalence of early febrile convulsions can be extremely high. But their occurrence is often short-lived, and no further problems occur after infancy. There are problems also of under-reporting for various reasons. Estimates from UK suggest a prevalence rate as high as 8/1000 in the general population, although a significant proportion of these have controlled fits (Saunders, 1984). In the Nigerian survey the prevalence was recorded as less than 1/1000 for grand mal epilepsy. In the Kibwezi survey the prevalence was 10/1000. The difference, of course, was that in Kibwezi a very effective epilepsy clinic and outreach service has been in operation for a number of years, and, for many of the persons identified in the survey, the condition was controlled. When a sound and well-established service is available, it is likely that under-reporting will be reduced significantly for all the obvious reasons. And this point illustrates the necessity for data to be collected from within an existing service structure, rather than as a pre-cursor to service development.

CONCLUSION

It is a myth that surveys lead to the development of services for disabled people. What will determine such developments, is the government's commitment of funds to the training of personnel and the creation of posts.

The problem with surveys is that they divert funds and human resources into a non-productive and time-consuming exercise. Often the argument used is that the funding agencies have money available specifically for research applications and if this is used for surveys, it will not detract from service development. We feel that such donors should be persuaded to allocate this spare money into service development, or evaluation of existing services.

Data is, of course, of vital importance, but we have argued that this is best obtained from the evaluation of an existing service when the needs of disabled people can be met appropriately, as, and when, they come to light. Updated 2001

REFERENCES

Baldwin, S, Stansfeld, P, Asindua, S, Gajjar, M, Wilkin, S (1989). Final report of the community-based rehabilitation programme: Survey of childhood disabilities in Kibwezi division, Kenya. Unpublished report, AMREF, Nairobi, Kenya.

Bernier, RH (1983) Prevalence survey techniques for paralytic polio: an update. Paper presented to the EPI Global Advisory Group, WHO, Geneva.

Davies, M (1982) National Survey of Disability in Zimbabwe.

DISTAT (1988) UN Disability statistics data base. UN Statistics Office, UN New York.

Newell, K, White, E & Gell, F (1987) The national sample study of prevalence of hearing-impairment in children in Swaziland. Unpublished report. Liverpool School of Tropical Medicine, Liverpool, UK.

Saunders, CA (1984) Handicapped children: an epidemiological study. In Curren, HV (ed) Nigerian children - developmental perspectives. RKP, London.

Saunders, CA (1985) A study of the prevalence of handicapping conditions affecting children, and a case finding intervention in the refugee population of the Gaza Strip. An unpublished report. Save the Children-UK: London.

Todd, C (1988) A report on the prevalence of iodine deficiency diseases in Lesotho. Unpublished report. UNICEF: Lesotho.

Ward, N & Mochaoa, B (1981) A survey to determine the prevalence of residual paralysis after poliomyelitis in Lesotho. EPI review, Lesotho Government Report/Save the Children-UK.

 


Fatal error: Uncaught Error: Call to undefined function mysql_connect() in /home3/eenet/public_html/reference.php:6 Stack trace: #0 /home3/eenet/public_html/resources/docs/surveys.php(201): require() #1 {main} thrown in /home3/eenet/public_html/reference.php on line 6