Omissions, inconsistencies, and red flags: Inside CNZ’s AOG assessment process

It's been over a month since Creative New Zealand announced the results of tiers 3 and 4 of its recently rejigged Arts Organisations and Groups (AOG) Fund.  

Outcomes were varied, and reactions were mixed. Whilst the dust may have already settled for some organisations, others are still coming to terms with what the future looks like and where their applications may have gone wrong. 

To better understand the results, The Big Idea made an OIA (Official Information Act) request into the process. What we found reveals that odds were always stacked against organisations and the assessment process lacked logic and transparency at multiple stages. 

To try and understand the reasoning behind the results, we've looked at the assessment process through CNZ’s internal documents (which differs in some detail to other process information they’ve published).  

Applications went through seven steps:  

1. Scoring 
Each application was assessed by two to four external assessors plus two CNZ staff members and scored against published criteria. They didn't see funding request levels. The maximum scores were 25 for Tier 3 and 30 for Tier 4. 

2. Artform panels 
The same internal and external assessors then collectively reviewed the strategic mix within each discipline and funding pool and recommended a funding priority (High, Medium or Low) for every application before making recommendations to a main panel. All applications were discussed at this step, but again the funding request levels of each organisation were not seen. 

3. Separate assessments 
Three sections of the applications – financial health and delivery to Hapori Māori and Pacific Communities – were scored separately and did not feed into the ranked list or panel recommendations. 

4. Main panel 
A main panel made up of CNZ senior leadership and a small group of external senior leaders checked the overall portfolio balance across artforms and funding pools. They didn't reassess applications or see funding request levels, made a handful of changes, and didn't discuss every application. 

5. Budget Allocation Working Group 
A Budget Allocation Working Group composed of senior CNZ managers supported by the CNZ finance team and other managers, then implemented the main panel recommendations by assigning funding and monetary amounts.  

6. CE sign-off 
The Chief Executive then endorsed the Working Group's recommendations. 

7. Arts Council sign-off 
And finally, the Arts Council endorsed the recommendations and funding levels. 

Across two tranches of OIA documents and multiple follow up questions with CNZ spokespeople we’ve confirmed:  

Assessors didn’t know how much funding each organisation was asking for. 
Individual assessors and assessment panels were asked to focus on “artform impact” alone and “did not receive information about the amount of funding requested”. Nor did they have a say over how much funding organisations should be offered. This omission placed an immense amount of responsibility on the Budget Allocation Working Group which CNZ confirms was the “only stage” in the process where funding amounts were considered. 

Funding decisions don’t always equate to assessor and panel rankings. 
CNZ confirmed “High Priority does not directly co-relate to the level of funding offered”. There are multiple organisations who carried the High Priority marker and high percentage scores all the way to the Budget Allocation Working Group stage and then landed in the Reduced tier. We don't know if any organisations were marked High Priority but received no funding at all. On the rating system, a CNZ spokesperson says: “We recognise there is no perfect assessment model, particularly in a highly competitive environment where the available funding is not sufficient to support every strong application.”

The Budget Allocation Working Group didn’t have a set methodology. 
CNZ confirmed there was “no prescribed funding formula that determined the amounts offered” and that “there wasn’t a single theme that determined funding decisions”. They provided a list of “factors” that the Budget Allocation Working Group used to make their decisions. This list is long and broad, so it’s difficult to pinpoint how it was applied across applications. Factors include “advised parameters about the level of contribution (generally not more than 70 percent of an organisation’s recent (2-year) average turnover), consistency/parity of investment with applicants currently funded under AOG and likely applicants to Tiers 1 & 2, previous annual funding levels via both Arts Organisations and Groups Funds and Investment Programmes (Tōtara and Kahikatea), comparability with other organisations, as well as funding provided via other Creative New Zealand programmes, sector and artform insight, experience and contextual knowledge and strategic considerations such as regional spread, artform coverage, breadth versus depth and organisational scale, as well as delivery to communities”. For organisations wanting more clarity however, CNZ has said: “We have offered, and continue to offer, meetings with organisations that want to discuss their funding decision in more detail. These conversations can provide further information about the assessment of their application, the feedback available from the process, and the broader context in which decisions were made”. 

Requested amounts became irrelevant. 
Organisations past funding levels were used as “a starting point” by the Budget Allocation Working Group because “funding based on amounts requested would result in very few organisations able to be funded.” When asked whether this implies the demand and/or ask was underestimated/oversubscribed, CNZ says “we did not want to predetermine the shape of the eventual portfolio by tightly constraining what organisations could request. Instead, organisations were able to make the case for the level of investment they considered appropriate to their circumstances, within the settings of the fund. The total amount requested, therefore, significantly exceeded the funding available”. 

They tested a flat increase approach, but there were not enough funds available. 
The Budget Allocation Working Group modelled a percentage increase applied evenly to all high-scoring, high-priority applications but confirmed it to be an “unsustainable starting point.” A CNZ spokesperson confirms: “The Budget Allocation Working Group tested several scenarios based on the recommendations they received from the panels. While demand and budget were known, the Working Group responded to the rankings and recommendations in front of them with a number of options”.

Some organisations were round pegs in square holes. 
Assessment panels raised concerns about whether some organisations were placed in the right artform category (particularly those whose work doesn't fit neatly into a single discipline) following the discontinuation of CNZ's former "Interarts" category. In response CNZ says that while it continues to review its funding approaches, “there is currently no intention to change artform categories for future funding rounds”. 

CNZ are yet to release how many organisations applied in total. 
We know 94 organisations were successful in attaining funding in Tiers 3 and 4 but we don’t know how many were unsuccessful despite multiple requests. This makes it difficult to know whether we’re only seeing the tip of the iceberg when it comes to the results and repercussions. Withholding this information is at odds with past reporting, but CNZ says this fund is “structured differently from previous funding programmes, including the way applications are grouped and how results are being reported. For that reason, releasing a single figure part-way through the process would not necessarily provide an accurate or useful comparison”. It says once the full round, meaning all tiers, are complete, “we will publish information and summary statistics for the fund as a whole, as we do for all our funds, including the total number of applications received and information about regional, artform and community representation”. 

Some of the application questions didn't count. 
“The Hapori Māori, Pasifika Engagement, and Financial Health questions were considered as part of the overall assessment process but were not included in the numerical ranking score” nor were they considered by the artform panels despite making up a large proportion of the application.

Some organisations might have been better off if they'd applied to a lower tier. 
30 of the 94 funded organisations ended up receiving less than $125,000 (technically Tier 1/2 territory). CNZ's own decision-making paper states: “It is notable that, should these organisations have applied for lower amounts, they would have likely been successful at Tiers 1 and 2.”

Assessors on panels felt unsure about priorities. 
Panel discussions flagged gaps in representation across certain regions and cultural categories. Some assessors wanted to undertake further discussions around whether they should be backing organisations with an established CNZ funding history or prioritise regional spread. CNZ states “We… look at representation and reach across our investment as a whole, rather than expecting any one part of a funding round to achieve a perfect balance across every artform, region or community.”

When reading through the documents, it's clear that this wasn't an easy process and CNZ knew it – its comms focus was on “being prepared and transparent rather than trying to avoid criticism”. In turn it proactively published information on the process, gave organisations advance notice before results went public, offered meetings and a webinar to affected applicants and briefed staff with “key messages and FAQs”. 

The Budget Allocation Working Group “registered discomfort” about offering organisations less than they'd historically received and displayed a desire to offer increases but were stumped by the limited budget. 

Assessors and assessment panels raised concerns around the lack of Tier 3 and 4 applications across specific regions and cultural groups which was reiterated by a CNZ spokesperson who says: “Where there are relatively few applications from a particular region, community or artform, there are correspondingly fewer opportunities to support organisations through that part of the process.”

The Main Panel questioned how much it should be trying to address imbalances in its own recommendations, including the balance between emerging and legacy organisations and the lack of Asian-led organisations. Another panelist raised that honouring Te Tiriti should be front and centre of every decision made. 

CNZ's own decision-making paper anticipated real consequences for organisations on the receiving end too, naming “reduced programming,” “workforce reductions,” and “increased financial vulnerability” as likely outcomes. CNZ's internal risk register listed “stakeholder dissatisfaction,” “public criticism,” and “increased scrutiny” as expected outcomes of the results. 

The documents show a process where the people involved clearly saw many of these problems coming and weighed them seriously, yet didn’t have routes through which to address or solve them meaningfully. 

On this, CNZ says “the level of demand meant there were difficult decisions to make about both which organisations could be supported and at what level. The fact that those tensions and concerns were recorded in the minutes reflects the care taken through the decision-making process. There were no easy choices: supporting one organisation, or maintaining an existing level of investment, necessarily reduced what was available elsewhere”. 

It also needs to be stated that at the same time the AOG fund was/is being tested at this scale, CNZ is in the midst of an internal restructure and the most significant decentralisation process in decades. Three major changes all happening within months and during an election year when their own funding has recently been cut. CNZ has a lot on its plate, has it bitten off more than it can chew?  

Since the AOG results were announced in July, concerns have been raised around a lack of set guidelines for the allocation of funding and there’s been a lot of finger pointing.

But this deep dive into process and decision-making isn’t about demanding a recount. Nor do we want to detract from the many well-deserving organisations who were successful in this funding round. We simply wanted to take a deeper dive into the process and understand the decisions made, in the hope that it might help shed some light and lighten the weight for those most impacted by the results, and ensure accountability for our sector’s biggest fund. 

But who's accountable when there’s so many factors at play and so much at stake?  

Over the last year, a new leadership team has stepped into the helm of CNZ for the first time in 19 years, and these aren’t fruitful times. There’s genuine external pressure and a shaky foundation so one could argue they deserve the benefit of the doubt. Still, they hold more responsibility than anyone, and their processes and communication need to be more open and transparent or they risk further isolating themselves from the communities they’re meant to serve.  

The Arts Council is worth scrutiny too. Whilst a balanced board is important, only a very small handful of members have backgrounds in the arts with most bringing governance, consulting, or executive experience from adjacent or unrelated sectors. Given they have final sign off, a clear understanding of the complexities of our arts ecosystem seems essential, however, this knowledge feels somewhat flimsy on the ground. 

Then there's Arts Minister, Paul Goldsmith, who now has 11 ministerial portfolios and claims recent budget cuts were intended for trimming administrative fat so more money could flow into the hands of artists. He’s gotten his wish and CNZ has cut a third of its staff, but at what cost? Are we now left with an under-resourced crown entity struggling to serve a sector doing it worse than ever?  

The election is less than two months away, and whilst a change of government is possible, it’s unlikely to equate to an increase in funding in the short and even medium term. Still, many hope a new Arts Minister may go a long way in stopping some of the bloodletting. 

CNZ has known for years that demand for its funding is increasingly outstripping the coffers, but in many ways, heads have been buried in sand. Now feels like the time to face some hard truths: help is not coming and pinning hopes on money that may not arrive for another 19 years isn’t a viable strategy.  

The task now is to start looking for alternative models and approaches outside of the sectors reliance on institutions like CNZ, whilst at the same time holding them to a higher level of accountability, so we can ensure what resources we do have are best utilised. 

With the results of a second AOG round fast approaching and the regional tender process ongoing, CNZ has a narrow window to look at current processes and see if improvements can be made.