Позитивные изменения, Том 3 №1, 2023. Positive changes. Volume 3, Issue 1 (2023) - Редакция журнала «Позитивные изменения»

TRANSFORMATION OF ACTIVITIES

The 2019–2020 evaluation resulted in a major transformation of all competitive procedures. The following key changes took place:

1. The competition is now part of the Foundation, not of its separate Road Home program.

The Foundation is implementing two special-purpose programs, but its activities are wider than the two Programs. In light of this, a separate target group was added: (child welfare specialists), and the grant competition has been expanded to accept projects aimed at crisis prevention work (media projects, public opinion-forming projects, projects to develop safe behavior skills).

2. The geography was also extended, first to the neighboring districts of the Vologda region, whose residents often commute to work in Cherepovets and whose children attend the educational institutions in the city. Second, a large regional center (Vologda) was also covered by the project. The third city added to the competition was Veliky Ustyug in Vologda Region, where Sveza company (an active supporter of The Road Home Program) has its office. Finally, Yakovlevo settlement in Belgorod Region.

3. Purposeful collection of feedback from applicants and experts has started, in order to make the necessary changes to the competitive procedures promptly. After the competition, we hold consultations and meetings with those applicants whose projects did not receive funding. Unsuccessful applications may be reworked together with the specialists from the Foundation’s Resource and Methodology Center and submitted to grant competitions by other grantmaking organizations.

4. The principle of variability of contest mechanisms was introduced: projects can be selected through an open competition, initiated by Foundation employees after analyzing the problems and the current situation in their territories, or the Foundation’s specialists can participate in partners’ projects.

5. The competition has received its own website: https://socialprojects.dorogakdomu.ru/. Applications are accepted only through the applicant’s personal account; experts also have to log in to review the applications.

6. The process of building the evidence base of the project starts during the preparation of the grant application (new sections have been added to the document).

7. The Foundation has defined the support functions for the competition (administrators, thematic consultants, programmers).

8. A three-tiered application review system has been tested and approved.

9. Independent experts are involved in the evaluation, which eliminates the conflict of interest for applications submitted by experts themselves.

10. A new body, the Competition Council was introduced.

Thus, as we can see, the evaluation has significantly increased the maturity of the competition, which also improved other processes of the Foundation: inter-departmental interaction mechanisms, positioning of the Foundation, planning of activities, project management (see Table 1).

Table 1. Immediate results of the Foundation’s grant competition after implementing the independent evaluation results

INTERNAL EVALUATION AND SELF-EVALUATION OF THE COMPETITION

These types of evaluations are conducted annually, following the principle of triangulation by data source. That is, information about the competition is collected from at least three data sources, to enable independent validation of the results and examine the evidence of social outcome from different perspectives.

1. Every year since 2008, discussions are held before and after the competition with the team of organizers. The analysis focuses on the following issues: directions and target groups of the competition, geography, and the contribution of grantees’ projects to the achievement of the Foundation’s strategic goals.

2. Since 2020, feedback has been collected annually from applicants and members of the competition committee. The data collection method is a blanket survey. An online questionnaire is used. In 2021, 66 % of the applicants and 81 % of the members of the competition committee participated in the survey. Here are some insights from the 2021 report on the feedback from the applicants and experts of the competition:

• “For the convenience of applicants and experts in 2020, reference materials were developed: a manual for the experts and a manual for the applicants. Almost all applicants and experts, with the exception of 7 % and 5 %, respectively, took advantage of the manual. All of them noted that the reference material was helpful when working with the competition site.”

• “60 % of the applicants received advice from the Foundation specialists by phone or email while preparing their applications. Communication with specialists was highly appreciated by the applicants and experts of the competition. Commenting on their evaluation, experts and applicants noted that consultations were received promptly, and the specialists were able to answer all questions of interest. Applicants who did not seek counseling say they were able to complete the application on their own, thanks to detailed instructions in the applicant’s manual. The experts thanked the entire competition team for their quick feedback, attentive and friendly attitude.”

3. Beginning in 2019, the professional community (evaluators, specialists of grantmaking organizations, and attendees of the evaluation conferences conducted by the Association of Program and Policy Evaluators) are involved in the discussions of competitive procedures.

FEATURES OF THE GRANT COMPETITION

1. The Road Home Charitable Foundation regularly conducts sociological and evaluation research on relevant issues. Research results are published on the organization’s official website and discussed in professional circles[56]. This ensures that potential applicants do not have to conduct a pre-project evaluation on their own, or go out looking for a problem or a vulnerable social group. The applicants can freely obtain this data from the research results.3

Information about the competition is collected from at least three data sources, to enable independent validation of the results and examine the evidence of social outcome from different perspectives.

2. Support to implementation of winning projects. Throughout the implementation period, the grantees are provided various kinds of support: information support (promotion, raising awareness, boosting the media coverage of the project); volunteer assistance; and methodological assistance.

3. Introducing an evidence-based approach at the application stage[57].

4. The competition has been extended to Cherepovets. Cherepovets is a medium-sized industrial city in Russia, which for several decades had the status of a single-industry city, with the city-forming