Treatment service users (TSU) project: phase two

6.6 AIVL reflections on demonstration projects as project administrator

Page last updated: March 2011

One of the significant distinctions between the TSU Project: Phase One and Phase Two is the fact that Phase Two focused on the implementation and evaluation of a series of demonstration projects whereas Phase One was a research project to map issues and attitudes in relation to consumer participation. The focus on implementing demonstration projects changed the nature of AIVL's role in the TSU Project: Phase Two from that of research investigator to project administrator. In contrast to the discussion points above, which specifically relate to the findings of the formal evaluation process, the comments below are a series of observations and reflections on project administration and working with the demonstration sites from AIVL's perspective. While these issues are not documented from an objective standpoint, they nevertheless contain important lessons for the future of consumer participation in drug treatment settings and therefore have been accorded an appropriate level of consideration in this report. In a number of places, the issues outlined below also have strong thematic connections with the findings from the baseline and evaluation interviews discussed above, and together this data forms an overall and realistic picture of the benefits and challenges of consumer participation in drug treatment settings. However, it should be noted that AIVL accepts the project sites may have a different and equally valid experience of the issues outlined below.

6.6.1 Involvement and role of drug user organisations
6.6.2 EOI process. project plans and implementation
6.6.3 Evaluation process
6.6.4 Expectations of the project

6.6.1 Involvement and role of drug user organisations

One of the more disappointing outcomes from the demonstration projects was the overall lack of genuine involvement and collaboration between the project sites and the relevant local drug user organisation. This is not to state that there was no involvement or relationship between these key stakeholders, but it is worth noting that when reviewing the project as a whole, no project site really made the most of this valuable resource, and the engagement that did occur with drug user organisations was sporadic. The reasons for this lack of engagement seem to mirror many of the barriers and challenges to involvement experienced by individual treatment consumers including structural and communication barriers.

Concerns about how structural and communication problems can act as barriers to consumer participation both for individual consumers and drug user organisations was identified in the TSU Project: Phase One. The National Consultation with Consumer Organisations in the TSU Project: Phase One highlighted the need to develop 'pathways to support more positive and constructive communication' between consumer organisations and services. They stated that such pathways could act as a strategy to address entrenched attitudes, identify and remove structural barriers and increase meaningful participation (AIVL, 2008, pp.73-74).

Taking this into consideration, all project sites in the TSU Project: Phase Two were required in the EOI process to address how they would work with the relevant drug user organisation in their state or territory. All project sites were able, in a theoretical sense, to outline how they would encourage such involvement. Subsequently, the requirement to engage with the local drug user organisation was also written into the service agreements with each project. The question that arises then is why was the engagement with drug user organisations so limited across the projects? Of the five demonstration projects only three sites (one in each of the three states engaged in the projects) had any contact with the local drug user organisation, and the engagement in one of those three sites amounted to only two meetings, initiated by AIVL, between the service and the local drug user organisation.Top of page

The reasons for the lack of user organisation involvement among the project sites that had little or no contact are likely to be explained with reference to existing roles and relationships between drug treatment service providers and service consumers. The TSU Project: Phase One highlighted the strong belief among many service providers that it was not 'the place' of consumers to have a role in decisionmaking with regard to service planning and delivery (AIVL, 2008, p.87). Similar sentiments were also expressed by service providers in the TSU Project: Phase Two. This attitude has also been highlighted in other studies relating to the culture of drug treatment services and how certain types of service culture can act to close off consumer roles and opportunities for service provider and consumer engagement (Treloar and Holt, 2006).

In this context, it is possible to envisage how such attitudes could be extended to relationships with drug user organisations and act as a very real barrier to engagement. If services do not support consumers taking a role in decision-making in relation to service planning and delivery, it follows that proactively seeking a relationship with the local drug user organisation may not be a priority for those service providers. It is possible this reluctance would be further underlined by the fact that the 'consumer relationship' in this case is even more likely than with individual consumers to result in consumer organisations asking for involvement in decision-making roles within the service. On the other hand, it is also possible that the problem of the consumer participation projects not being considered 'core business' by any of the services meant that there was not one staff member following up on contacts and liaison with the local drug user organisations and it therefore simply 'slipped through the cracks'. Either way, it highlights a problem with valuing the role and contribution consumers can bring to their services.

However, as outlined above, a number of project sites did include a reasonable degree of engagement with the local drug user organisation in their consumer participation projects. Of interest is the fact that both of these project sites had existing working relationships with the local drug user organisation. Indeed, one of the sites had already commenced discussions about another potential consumer participation project in partnership with the local drug user organisation prior to applying for the TSU demonstration project. From the evaluation data both the consumers and the staff in these services also seemed to have a greater level of awareness and understanding of consumer participation in the drug treatment context. This outcome is supported by available literature in the area of mental health and consumer participation which highlights the importance of 'practical experience' in changing attitudes towards the value or benefits of consumer participation, particularly in relation to consumers having a role in decision-making (Northern Area Mental Health Service, 2003).

While these sites were able to utilise existing relationships with their local drug user organisations as a platform for further engagement, it is noteworthy that even in these projects the involvement of the drug user organisations was not consistently sustained across the projects. The reasons behind this include the chronic turnover of staff in one service, which made ongoing involvement and stability of the relationship impossible, and a lack of resourcing in the drug user organisations which severely reduced their interest in and capacity to participate in a meaningful way in the project. While the negative impact of staff turnover is addressed elsewhere in this section, the lack of adequate resources to support the involvement of drug user organisations in the demonstration projects was a significant problem.

In addition to agreeing to involve drug user organisations in their demonstration projects, services also agreed to ensure that some of the available project funding would be used to resource drug user organisations for the roles they were asked to undertake within the projects. As outlined above, the majority of services did not engage the local drug user organisation in a meaningful way and therefore resourcing this role did not arise. However, a lack of resourcing did affect at least one of the projects and this example highlighted the importance of ensuring adequate resourcing for drug user organisations if they are expected to support the development of consumer participation in treatment services.

This is consistent with the findings in the TSU Project: Phase One which identified the lack of resourcing for consumer participation work as one of the 'main reasons why they struggle to effectively coordinate consumer participation efforts across the drug treatment sector' (AIVL, 2008, pp.89-90). Although the service involved did offer some resourcing for delivery of training for consumer representatives, the overall level of project funding meant that the service was not able to offer the level of resourcing needed to cover the 'real' cost to the drug user organisation of designing and delivering the training. At the heart of this problem is the fact that the vast majority of drug user organisations do not receive 'core capacity' funding for consumer participation work and this makes it very difficult for these organisations to prioritise participation in projects conducted by individual treatment services. In this regard, what can appear to be a lack of interest in participating from drug user organisations can in fact be a lack of ability to participate due to resourcing limitations.

In the end, regardless of the reasons, all five demonstration projects suffered from a lack of sustainable involvement from the local drug user organisation. The significant role that consumer organisations have played in increasing access to and uptake of health services and improving health outcomes for Australians is well documented (Consumer Focus Collaboration, 2001). A consistent theme across the TSU Projects: Phases One and Two is an ongoing reluctance to involve drug user organisations in consumer participation activities within the drug treatment sector.

This is further compounded by side discussions about which consumers drug user organisations can or do represent, and how to address the needs of those consumers who are not thought to be represented by such organisations. However, it is up to drug user organisations to decide who they purport to represent. Whether this is accepted by others or not is a separate matter, but it does not change the way that drug user organisations view themselves or those they seek to represent. Too often it seems that discussions about the diversity of treatment consumers are used to avoid taking action on the central issue — that with proper resourcing, existing drug user organisations would have the capacity (and view themselves) as the appropriate entities to represent treatment consumers with illicit drug dependencies. The apparent absence of a group to represent alcohol treatment consumers does not diminish this fact. It is time to move away from continual and unproductive discussions about the 'absences' and 'gaps' in drug user organisations to an acknowledgement of their role and value. In short, it is time to acknowledge and make use of what is there rather than focusing on what isn't.
Top of page
While there were some exceptions, the majority of treatment services involved in the TSU Project: Phase Two chose to conduct their consumer participation projects without the meaningful involvement of local drug user organisations. This was the case despite:
  • All projects experiencing problems with accessing and engaging their service consumers;

  • At least two services experiencing problems recruiting suitable consumers to take up paid consumer representative positions; and

  • A number of services complaining about a lack of resources and skills among existing staff to support consumer representation and the project generally.
The above issues are significant as they are all issues that could be successfully addressed by establishing and adequately resourcing an effective working relationship with the local drug user organisation.

This outcome reflects an increasingly urgent need to build greater awareness, understanding and, most importantly, acceptance among services and funding bodies of the role that drug user organisations can and must play in building consumer participation in drug treatment settings. Ultimately however, this will require a change in the culture of treatment services to allow consumers and, by extension, their representative organisations to take an active role. Other areas of health service delivery have managed to bridge this gap with very positive results for both service providers and service consumers (NAMHS, 2003). In this regard, it is positive that at least three services mentioned in evaluation that the experience of other sectors in consumer participation, particularly mental health, could provide useful lessons for the drug treatment area. That said, addressing the issues and concerns outlined above will require transformation at the level of both policy and practice.

Finally, it should be noted that transforming the relationship between drug treatment services and drug user organisations will require good faith, trust and commitment from all parties. The TSU Project: Phase Two evaluation data reveals that meaningful collaborations between treatment services and drug user organisations not only results in a greater awareness of the potential benefits of consumer participation for all involved, including consumers, but also has a positive effect on how consumers view the work and role of drug user organisations. Some consumers expressed the view that involvement in the consumer participation project at their service not only raised their awareness of their local drug user organisation, but also provided a 'way into' the drug user organisation as they could now see the tangible benefits of being involved.

While the demonstration projects funded for the TSU Project: Phase Two did not necessarily achieve the level of project outcomes specified in the original project plans and logframes, overall the project has begun the process of identifying both the barriers and incentives to consumer participation projects in this setting. With both treatment services and drug user organisations having a good deal to gain from collaboration in this area, the future of consumer participation in drug treatment services will rely on the development of effective working relationships between these two key stakeholders. Despite some existing relationships, by and large this work still needs to be undertaken.

6.6.2 EOI process, project plans and implementation

The TSU Project: Phase Two was unique in that it focused on the implementation and subsequent evaluation of five consumer participation demonstration projects in a range of drug treatment settings. The process for selecting the demonstration sites involved an advertising and EOI process followed by an assessment and selection process. This backdrop to the projects is important largely because it goes to the heart of some of the issues experienced by AIVL as the project administrators.

The evaluation data reflects an inconsistency across all projects in relation to the agreed project plans and logframes compared to the actual outcomes and achievements of the projects. While AIVL received a good level of interest from drug treatment services when the advertising and EOI process was conducted (and found services to be co-operative and enthusiastic when negotiating and finalising the project plans), it subsequently experienced a range of complications with the projects from a project administration viewpoint.

High staff turnover in key project positions caused long delays to one project and, from a project administration perspective, this appeared to be further complicated by the lack of an adequate handover to new staff. This not only caused further project delays but also resulted in confusion and misinformation among consumer participants in relation to project activities and arrangements. The impact of this situation can be seen in the evaluation data which reflects a 'change of heart' from consumers between baseline and evaluation data collection as their initial enthusiasm and commitment to the project wanes due to a drop in project momentum and confusion and delays in agreed consumer payments.

While the service acted to address the staff vacancies as quickly as possible, these types of problems are part of the reality of conducting consumer participation projects within large treatment services. Consumer participation activities are unlikely to be well-resourced or even resourced at all within most treatment services. Inevitably, staff will move on to new opportunities creating vacancies not only in their primary position but in the 'associated' roles they carry, such as consumer participation projects. In the end, the root of the problem may be associated, once again, with the overall lack of adequate resourcing and philosophical support for consumer participation activities within treatment services rather than what appears to be the problem on the 'surface' — that is staff vacancies.
Top of page
The funding for the demonstration projects as a whole was insufficient and meant that projects were only provided with a 'contribution' towards project costs from AIVL. As the project administrators, AIVL was restricted in the funding that could be provided to services by the amount of overall funding received for the TSU Project: Phase Two. AIVL was clear at the EOI stage that the funding available for demonstration projects was a 'contribution towards project costs'. However, in offering this contribution there was an expectation that services would already have or would be in the process of establishing their own investment in consumer participation as an essential rather than 'optional' aspect of a quality treatment framework. Indeed, all of the services selected for the TSU Project: Phase Two identified in their EOI that they had already made some level of commitment to consumer participation within their service model.

A significant part of the rationale for this approach was that the TSU Project: Phase One found that engagement in lower level consumer participation activities (according to the definition and model adopted for the project) was relatively commonplace among drug treatment services (AIVL, 2008, p.84). Therefore, it was agreed that the TSU Project: Phase Two would focus on services that were ready to move beyond lower level participation activities. On this basis, the TSU Project: Phase Two assumed a level of existing investment in consumer participation and presented services with an opportunity to 'start-up' higher level activities or 'value-add' to their existing consumer participation activities. It would not, however, provide the basis for large-scale projects that were commencing from 'day one'. The concept of providing extra funding to support services to 'start-up' consumer participation activities was also strongly identified in the TSU Project: Phase One Report (AIVL, 2008, p.89).

It is interesting to note that comments from services in the evaluation data continued to identify the need for 'start-up' resources to build support and commitment for consumer participation before being able to embed it as a standard practice within the organisation. Given that all of the services were provided with funding as part of their participation in the TSU Project: Phase Two to value-add to existing and 'start-up' higher level activities, it must be asked whether resourcing problems are the only barrier to embedding consumer participation in drug treatment settings. Perhaps it is possible that although a lack of resourcing is a significant part of the problem, it is not the total picture. For example, are there problems at the policy or structural level that are preventing meaningful consumer participation being put into practice in the drug treatment context despite a level of resourcing being made available?

In evaluation some services identified the importance of flexibility in project administration as a very important factor in allowing under-resourced treatment services to adapt projects as needed and in response to client needs. While AIVL agrees with this principle and did attempt to provide a flexible environment for project implementation, concerns emerged across most projects in relation to the seeming lack of involvement of consumers in project planning and a lack of response to progress reporting requirements. Despite requests for update reports to allow AIVL as project administrator to provide targeted support and trouble-shooting for projects, only one of the five projects provided any of these reports.

AIVL had attempted to keep the progress reporting for the projects to a minimum — requiring only a brief update on progress after the first month and one at another point across the six-month projects. In addition, as AIVL was aware of the negative impact onerous reporting requirements could have on the small-scale projects, the request to provide progress reports was not included in the logframes as a formal project outcome but rather as a more informal agreement with each of the project sites. The AIVL Project Officer contacted the sites as appropriate, based on their commencement date and relative progress, to seek brief, written progress reports. While sites were willing to provide a verbal update of progress, only one site complied with the request for the written updates.

Given the numerous comments in evaluation relating to services being under-resourced and understaffed, it is possible that projects found it difficult to justify and/or prioritise progress reporting for a relatively small amount of funding. The negative impact that unnecessary reporting can have on project outcomes and the need to ensure reporting does not place a disproportionate burden on small projects and organisations has been well documented in numerous studies within the health sector (New South Wales Council of Social Services, 2009). In this regard, it is a legitimate concern for services implementing small-scale consumer participation projects. Nonetheless, if this was a problem it needed to be raised with AIVL and addressed through a renegotiation of appropriate milestones rather than by neglecting or refusing to report.

The lack of regular progress reporting meant that in a project where there were five different sites across three states, AIVL was effectively locked out of the support role it could have provided to services in the implementation of their projects. AIVL's budget, too, was extremely limited and without funds to regularly visit the projects, and in the absence of written progress reports, a 'disconnect' developed between the projects and AIVL as the administrators. While AIVL accepts responsibility for its part in creating this breakdown in ongoing communications, it also needs to be acknowledged that the project sites did not make active attempts to contact AIVL and address this situation.

Verbal reports were useful when provided but this relied on the ability of the AIVL Project Officer to reach relevant contact people at each of the sites within a reasonable timeframe. As the AIVL Project Officer was a part-time position and services are busy and under-resourced, this was often difficult. Verbal reports also increased the chance of important issues being lost in translation or overlooked as they were communicated from the project site to the AIVL Project Officer, and then onto other AIVL staff and staff from the NCHSR. When this issue is coupled with the general lack of engagement with local drug user organisations identified above, the result was an unfortunate dismissal of a good deal of expertise and support in relation to consumer participation that could have been provided by AIVL and its members.

In addition to the concerns about reporting, the evaluation data raises questions in the majority of cases about the level of consumer involvement in the development of the project plans at any stage of the process. Statements by services admitting a lack of genuine involvement of consumers when planning their projects and referring to consumers as typically being engaged at the 'rubber stamping stage', raise concerns about the quality of some of the projects conducted. It is accepted that consumer participation in drug treatment settings can mean it takes time for understanding, relationships and trust to be established. In this context it can be difficult to find consumers who are interested and willing to act as consumer representatives. However, what is less clear is why the opportunity to revise project plans with genuine consumer input was not taken up after the project commenced and once a number of key consumers were involved?Top of page

Evaluation data from both consumers and staff reflected an absence of consumer involvement in key project activities, particularly those in the mid- to high-level range involving decision-making roles such as having input into the job descriptions for consumer representative positions. This resulted in many consumers having very limited knowledge of the project, and many framing the project outcomes more in terms of individualised self-help and support rather than as active involvement in service planning and decision-making. In some services, this view was reinforced by the belief of some staff that there are certain roles that are 'staff only' and 'inappropriate' for consumer involvement. The view that there are certain roles that are 'inappropriate' for consumer involvement was also a theme expressed by some services in the findings from the TSU Project: Phase One (AIVL, 2008, pp 57-60).

While it is acknowledged that in all health-care settings there are roles and responsibilities that are appropriately restricted to trained professionals, the roles being referred to in relation to these projects do not fall into this category. They are roles and responsibilities that many other areas of heath service delivery have designated as not only appropriate to be undertaken by consumer representatives but frequently result in better outcomes for both consumers and services when undertaken by or with consumers (Norman, J et al., 2008). These roles include input into project plans, consumer representative position descriptions, service planning committees, staff training sessions, service policies and staff recruitment, etc.

In addition to the lack of consumer involvement, in many cases th6e projects had also been developed without the input of the majority of staff. Evidence of this lack of staff involvement was highlighted in the evaluation process and is discussed further in Section 6.6.3 below. In some cases, services did not have effective strategies for encouraging and supporting communication among staff. This suggests that many of the services were not well-placed to embark on medium-level consumer participation despite indications to the contrary in the initial project proposals. Developing clear and effective ways to communicate with staff and consumers within services is likely to be central to successful consumer participation and, therefore, 'pre-consumer participation activities' aimed at developing engagement and communication strategies among consumers and staff should be first priorities.

It should also be noted that all services involved in the TSU Project: Phase Two had at least one key staff member who was committed to and interested in consumer participation within the service. By and large it was these staff members who drove the EOI process and took responsibility for developing the project plans, negotiating the logframes and liaising with AIVL and other staff within their service. Despite the best intentions of a few enthusiastic staff in the services involved, much of the activity being labelled as 'consumer participation' over the course of the evaluation was shown to be largely staff-driven project activity designed primarily to meet accreditation standards, internal evaluation needs and externally imposed performance measures. Some projects had more genuine consumer involvement than others but no project had a strong sense of consumer ownership or being truly 'consumer-driven'.

The lack of consumer ownership over the activities and outcomes of the majority of the TSU Project: Phase Two demonstration projects is highlighted by the lack of awareness and understanding among consumers identified in Section 6.2. A few understood the purpose of the evaluation process but most did not and thought that the evaluation was an evaluation of the service, not of the consumer participation projects. Outside of a very small number of consumers who were employed in paid consumer representative roles, the majority of consumers interviewed had little or no knowledge of the consumer participation projects overall, let alone any involvement in developing the project activities and outcomes.

This situation highlights the level of training and capacity building that is needed among staff and consumers but, importantly, also among senior management and funding bodies in relation to consumer participation. Such training could support an understanding of consumer participation as a fundamental activity in its own right — with its own benefits and value — not just as a way to meet service accreditation and/or funding outcomes and deliverables. The need to reposition or shift the mind-set about consumer participation as being an 'optional extra' or done as a 'favour' to consumers was another underlining theme identified by AIVL and backed up by the evaluation data. In some of the services there appeared to be a lack of fundamental support and engagement in the project from senior management resulting in a general deprioritising of the project. This was reflected in comments from staff about a lack of support for consumer participation as 'core business' within the service, staff writing project plans in isolation, and little or no evidence of action taken by management in relation to 'white-anting' or outright refusals to participate in the projects by other staff.

Consumers interviewed in evaluation also identified this important issue. Consumer representatives involved in one project with high levels of staff turnover recalled the project as feeling 'rudderless' at times due to the overall lack of organisational investment in the project. These comments indicate a belief that the negative impact of staff turnover during a consumer participation project could be minimised, providing there is a strong commitment from senior management to ensure consumers feel supported and valued while new staff are recruited. Consumers also highlighted the potential for even enthusiastic consumers to gradually lose interest if there appeared to be little progress within the project. This is a major concern given that both consumers and staff identified the difficulties associated with getting consumers interested and involved in ongoing consumer participation roles as opposed to participating in one-off surveys or informal discussions.

These experiences are contrasted with the comments from consumers involved in one of the projects with a good level of senior management and organisational support. In this project, consumers identified the importance of staff support and feeling trust and commitment at a service or organisational level. As identified in Section 6.6.1 above, drug user organisations could play an important role in building an understanding of the potential benefits and value to the service of consumer participation. Interestingly, the above project where consumers identified a high level of staff and management support was one of the demonstration projects that involved the local drug user organisation in the development of the consumer training for the project.

From the vantage point as Project Administrators, AIVL strongly identified the need for organisational change and capacity building in relation to consumer participation for most services. Very few staff at baseline identified the need for management or staff training in this area, however there was some shift in this view at evaluation. This shift supports the view articulated in the TSU Project: Phase One and consumer participation projects in other areas of health service delivery that an understanding and valuing of consumer participation often only comes after services have had some practical experience of engaging with consumers (AIVL, 2008, p.25 and NRCCPH, 2004).

In some cases, the existence of consumer charters of rights and responsibilities and/or quality frameworks requiring consumer participation were seen as sufficient evidence of organisational commitment to consumer involvement. Others, however, recognised the need for cultural change within their services before such policy frameworks would result in the meaningful involvement of consumers. The TSU Project: Phase One also showed that the mere existence of policies identifying the need for consumer participation did not necessarily result in a valuing of consumer participation by the service in a practical sense (AIVL, 2008. pp.85-86).Top of page

6.6.3 Evaluation process

As the Project Administrators, one of AIVL's roles was to organise the logistics for the evaluation process. This involved contacting the demonstration sites to arrange two rounds of data collection — at baseline and then at evaluation towards the completion of the project timeframe. The data collection methodology is outlined in full in Chapter 4 of this report. A number of issues were highlighted during the process of coordinating the data collection which, although largely administrative in nature, have implications for the larger themes explored in other parts of this section.

Despite the fact that each of the project sites had designated and agreed project contact staff, at the time of organising the baseline and evaluation data collections AIVL experienced difficulties in making contact with these staff in a number of the sites. As identified above, the TSU Project: Phase Two had limited resources and for this reason AIVL aimed to provide a degree of flexibility for the project sites in relation to the timing of their data collection. However, there were important ethical obligations guiding both AIVL and the NCHSR to ensure that all data collection was conducted in accordance with strict ethical guidelines for evaluations of this type.

While two project sites met all evaluation requirements and responded in a timely fashion to all AIVL requests to schedule data collection points, unfortunately AIVL experienced moderate to significant delays in the three other project sites. It should be noted that one project site in Victoria experienced unforeseen delays to their project associated with the Victorian bushfires, which also affected scheduling for the second data collection point. However, this project site still worked effectively with AIVL to schedule all data collection points in a timely manner. Despite this, delays in contacting and therefore scheduling the evaluation points at the three other services ultimately caused irreparable damage to the overall timeframe for the TSU Project: Phase Two and resulted in AIVL having to seek a variation to contract to extend the project and deliverables. Some of the delays and problems included the inability to make contact with project staff via email or telephone for more than four weeks; scheduling data collection points to have them cancelled on the day AIVL and NCHSR staff were due to travel; and ongoing delays to scheduling of data collection points due to constant turnover of project staff.

In addition to the above problems, a number of concerns also surfaced once AIVL and NCHSR staff arrived at the project sites for the scheduled data collection. At all but one project site, staff at reception or other first points of contact did not seem to be aware of the project or know who the relevant contact person was for the project. Further, the majority of staff at two project sites also seemed to have little or no awareness of the scheduled evaluation. These occurrences raise a number of questions about the consistency and quality of the projects in these sites, including why were so few staff aware of the projects, and what were the implications for the ongoing progress of the project if the single responsible staff member needed to take leave or resign from their position?

These questions were somewhat answered by the fact that the project experienced a high level of staff turnover, with at least three different project staff. In evaluation, consumers involved in this project expressed highly divergent views about their involvement in — and the quality of the project between — baseline and evaluation data collection. At baseline consumers were enthusiastic, committed and expressed a sense of empowerment about the possibilities of the project. In evaluation, the consumers were difficult to locate for interview and those who did come forward expressed disappointment with how the project had been managed, the lack of continuity and the fact that commitments made to consumers about payment and work had not been honoured.

At baseline and evaluation, five consumers were interviewed from each of the demonstration project sites. The project sites were asked to provide five consumers, preferably all directly involved in the project, for interview by the evaluators. Of concern was the fact that all project sites seemed to have difficulty in finding five consumers for the evaluation interviews who had been directly involved in the consumer participation projects. Although most of the project sites were able to provide one consumer, often a paid consumer representative, to interview specifically about the demonstration project, no site was able to provide five or even a majority of consumers who had been involved in the project. Of the consumers who were interviewed, a majority at all project sites seemed to have little or no knowledge of the project.

While these issues have also been explored above, they require some further comment as they do have implications for the quality of evaluation. The evaluations took longer than expected due to having to return to one site for follow-up interviews and having to take extra time to explain the focus of the evaluation to consumers. This would no doubt have been different if consumers who were directly involved in the projects could have been interviewed. Although the reasons for not being able to provide relevant consumers for the evaluation interviews varied from site to site, the major reasons included that the project had not yet properly commenced or interviews with appropriate consumers could not be arranged as services had found it very difficult to access and involve consumers in the projects. While not seeking to make 'excuses', one of the project sites also highlighted the difficulties associated with recruiting consumers to be involved in ongoing consumer participation projects as opposed to one-off surveys and focus groups.

From AIVL's perspective as Project Administrator these concerns also linked to the issue outlined above in relation to consumer involvement and ownership of the demonstration projects. While AIVL fully acknowledges the barriers to conducting consumer participation projects in drug treatment settings, the difficulties in accessing informed consumers for the evaluation interviews at the very least raise questions about the priority given to these projects by the services involved and the quality of the engagement that occurred.

Finally, it should be noted that both AIVL and the NCHSR also experienced staff turnovers during the TSU Project: Phase Two. While staff changes during time- and resource-limited projects always create logistical and strategic challenges, in both cases replacement staff were located, briefed and in place with little or no impact on the overall timeframe and with no significant break in contact with demonstration project sites. The major negative impact of the AIVL staff changes on the overall project was that email updates to the PAC and monitoring calls to the project sites were less frequent than planned. While this was not ideal, it did not affect the implementation of the demonstration projects and both face-to-face PAC meetings were held at the planned project points — at project commencement and at draft final report stage.Top of page

6.6.4 Expectations of the project

Following on from the issues raised above about the degree of ownership over the projects by services and consumers, this final section explores a range of issues in relation to project expectations. One of the key issues for AIVL as the Project Administrators was the degree to which the demonstration sites took responsibility for their projects and their outcomes at evaluation. As noted above, demonstration sites were selected based on an EOI and independent selection process. The interest in conducting the consumer participation projects was more than twice the amount that could be funded within the project budget. The result was that AIVL had to unfortunately reject numerous EOIs from enthusiastic and highly regarded drug treatment services as only five projects could be selected.

This background context is very relevant as a number of concerns have emerged across the project, in many but not all project sites, about the need to improve:
  • The level of senior management commitment to consumer participation;

  • The overall culture and environment in drug treatment services to better support consumer participation; and

  • The degree of involvement and therefore ownership of consumers in consumer participation projects in drug treatment settings.
Despite applying to conduct the project and putting considerable effort into developing project plans and negotiating the project logframes, AIVL did not experienced the same level of commitment from a majority of the projects in the project implementation phase. At certain points in the project implementation, AIVL was forced to reiterate to two of the sites that the demonstration projects were not 'AIVL' projects but their projects. It seemed at these two sites, at least, there was an expectation that after being selected to conduct their project, AIVL would work with them directly on the implementation phase. Not only was this inappropriate as AIVL is a national organisation, but also because AIVL would be playing a strong role in the evaluation of the projects through the consumer interviews. It was this separation of roles that was behind the rationale to require all project sites to work in collaboration with their local drug user organisations. Such arrangements would have established sustainable, locally relevant relationships between consumers, consumer organisations and drug treatment services.

AIVL also had to manage and modify its expectations of the project outcomes during the course of the demonstration projects. As already outlined above, all of the sites selected were required to demonstrate their existing commitments to low-level consumer participation activities and their interest in utilising the TSU Project: Phase Two to build on and expand these activities. The services not only offered different types of treatment services but they also had different levels of experience and development in relation to consumer participation and involvement. From the project plans submitted and the baseline interviews it was clear that some services had a higher level of senior management support for their project, existing relationships with local drug user organisations, and more established communication channels with their consumers.

In this context it was surprising that none of the projects seemed able in evaluation to fully demonstrate that they had achieved all or even most of their original project outcomes. Why was it that even those services that seemed to have some of the key indicators for successful consumer participation in place at project commencement were still unable to translate this advantage into clear and tangible project outcomes? Certainly, staff in some services claimed to have achieved significant results from their projects with staff from one service remarking on the fact that one client was so committed to the consumer participation project that they extended their stay in the service to continue their involvement. Regardless of this situation, this was not one of their project outcomes and for this service, and all the other project sites, consumers across the board did not verify the degree of activity and outcomes that key staff outlined in evaluation interviews.

AIVL acknowledges that there may be differences in the way that staff and consumers both perceive and articulate project outcomes. It is also acknowledged that not all consumers interviewed at the evaluation stage contradicted the statements and views of service staff. However, the difficulty is that the differences greatly out-number the similarities and they are significant and point to fundamental problems with communication between consumers and services. These communication barriers seem to be heavily associated with the existing roles and relationships between service providers and consumers. It seems that even where there is willingness to explore these roles, the TSU Project: Phase Two demonstration projects were not able to break through these barriers despite the best of intentions. This is supported by many comments made by both parties in evaluation, with staff acknowledging entrenched attitudes and power differences, and consumers expressing hope for a breakdown of these barriers and differences. Once again, these same issues and hopes were also articulated in the findings of the TSU Project: Phase One (AIVL, 2008, pp. 87-89)

Where does this leave the ideal of consumer participation in drug treatment settings? Is it impossible to genuinely involve consumers of drug treatment services — regardless of the service type — in the decisions about their service and their treatment? Are the treatment service models incapable of coping with the openness required for such engagement to work? Although it could be argued that some, even many, of the conclusions drawn about the demonstration projects in the TSU Project: Phase Two could indicate that consumer participation in illicit drug treatment services is not possible, this conclusion seems wholly unsatisfactory.Top of page

While all the services were committed to their demonstration projects, in practice many struggled to implement their projects and have not achieved the outcomes that they or AIVL had hoped would be achieved. However, it is important to acknowledge that these services have, for the first time, opened themselves to what it might take to make consumer participation in this context a reality. It is promising that the project sites identified in evaluation that they had underestimated the demands on the organisation, particularly at the 'higher levels' and that they needed to do further work on developing strategies for engaging consumers outside of the therapeutic or clinical environment. It is also important that a number of the services stated they were surprised and encouraged by the positive feedback and enthusiasm of consumers and that this gave them encouragement to keep moving forward.

This was the first national project to conduct a series of demonstration projects to support consumer participation in drug treatment settings. Many problems and barriers have been uncovered. We cannot pretend these barriers and problems are not present simply because we wish it so or because it might upset some services, staff or consumers. However, it is possible to take these challenges and commit to the process of dismantling and addressing them one by one if necessary. Some of them are significant challenges for the drug treatment sector. Changing the culture of drug treatment services to open up communication and trust between staff and consumers will not be easy, but any steps in that direction will open the possibility of consumers having a stronger voice in the future. Some services are already making steps in this direction, but the need to continue the steps towards consumer participation in drug treatment services is probably articulated best by a consumer from one of the demonstration projects, who said:

'I think what I'm hoping it will achieve is a common ground for staff and consumers to work out things, like you know, how do you put it into words? It's like an even plane so everyone's on the same, like no-one's better than anyone, no-one's — even though this person works here and you're a consumer, you both got the same rights and you're both equal. You don't have to sit there, you know put your head down and think "oh they're the staff, I can't say anything what if they take my kids? What if they do this? What if they do that?" You know...'