{"text_format":"1","workbook":"xlsx","n_sheets":14,"sheets":["User guide","Structure","Summary Page","Documentation","Structure & Clarity","Verification","Validation","Data & Assumptions","Model score","QA plan","QA history","Issues Log","Version Log","Lookups"],"not_rendered":{"charts":2,"images":1,"macros":1,"external_links":0,"defined_names":81,"comments":1,"pivot_tables":0},"unopened":[],"bounds":{"document_bytes":16777216,"sheet_cells":100000,"rows_first":2000,"rows_last":500,"rows_between":2000,"halvings":0,"cell_chars":16384,"cell_head_chars":4096},"cells_cut":0,"cells_raw":0,"read":"whole","date_system":"1900"}

{"sheet":"User guide","index":0,"n_rows":36,"n_cols":11,"rows_kept":36,"cut":false,"hidden":false,"merged":[[5,1,5,6],[7,1,7,6],[9,1,9,6],[28,1,28,6]]}
[null,null,null,null,null,"Excel models"]
[{"$f":"IF(SecurityClassification=\"OFFICIAL\",\"\",SecurityClassification & \" - \")  & DocumentName & \" QA log - version \" & VersionNumber","v":" -  QA log - version "}]
[]
[null,"QA LOG SCOPE"]
[null,"What / why?"]
[null,"The QA log is a list of assurance activities such that if carried out they provide confidence to the model developer(s), assurer(s) and user(s) that a given version of the model is fit for purpose (with 'purpose' defined in the model scope documentation). It should be easy to assess the level of confidence in the model through the scoring system (models should aim for 90%), and to clearly identify omissions or areas for improvement so that analysts and decision makers can judge the risks of using outputs from the model as it stands."]
[null,"Who is responsible for the log? "]
[null,"The main model developer must ensure that the log is up to date. All in-house BEIS models must have a log, as must all new 3rd party models.  When working with a 3rd party you should agree who will be responsible for maintaining an up to date log within the 3rd party or BEIS. Log completion for existing 3rd party models should be judged on a case-by-case assessment (based on size / complexity / business criticality etc). All QA work completed should be recorded; including by whom, their specialism / authority, and the review date. It is mandatory for at least one person in addition to the model owner to carry out the checks."]
[null,"When is it used?"]
[null," It should be completed during model development by the developer and post-development by those performing QA work. It should be sent alongside any analysis that goes for clearance as it can be used to assess the risk associated with any evidence generated by the model."]
[]
[null,"USER GUIDE"]
[null,"What to do"]
[null,"1) Check this is the correct QA log version. This version should be used only for Excel models. If your model is not developed in Excel, please use the alternative non-Excel version"]
[null,"2) If the QA log has been already filled in, check that the previous QA review information has been recorded in the 'QA history' sheet. If not, press the 'Update QA History' button in this sheet."]
[null,"3) Complete the summary page with the required information in the white boxes. It is fine to overwrite if the previous information has been recorded."]
[null,"4) Complete the data required in the 5 orange tabs. There is guidance on how to do this below. It is fine to overwrite if the previous information has been recorded."]
[null,"5) Use the Issues Log sheet to log the outstanding issues in the model."]
[null,"6) After all the relevant cells have been filled, press the button in the 'QA history' sheet to record the information."]
[null,"7) Let your QA contact at BEIS know that the QA log was created / updated."]
[null,"8) Provide feedback to us on the QA log (if relevant)."]
[null,"What not to do"]
[null,"1) Change the structure of the sheets (e.g. add new columns, rename table headers...)"]
[null,"2) Leave blank spaces without explaining why."]
[null,"3) Create a new QA log for a model that already has one. Every model should have one QA log and record all the quality assurance work there."]
[null,"Legend"]
[null,"To be completed by user","Not to be changed"]
[null,"Notes"]
[null,"1) If the QA log is used outside BEIS some of the links provided may not work as they refer to internal documents. For any issues contact the Modelling Integrity Team. \n2) The log is protected to avoid changes in the structure. All the relevant fields are not locked and can be filled also in protected mode. If for sensible reasons you will need to change the structure contact the Modelling Integrity Team and you will receive an unprotected version.\n3) QA should be proportionate to the model complexity and importance. The QA tasks described in the orange tabs are appropriate for complex, business critical models; if the model is simple and not crucial for the department you may consider a lower level of QA. Contact the MIT if you have any doubts\n4) For any issues or questions please contact the Modelling Integrity Team at DECCModellingintegrityTeam@beis.gov.uk"]
[null,"BEIS contacts"]
[null,"Webpage for QA files","https://www.gov.uk/government/collections/quality-assurance-tools-and-guidance-in-decc"]
[null,"Email address","DECCModellingintegrityTeam@beis.gov.uk"]
[]
[null,null,"Task Code","QA requirement","Guidance and links","Performed by","Rating","Rating guidelines","Current status (issues found)","Evidence (when applicable)","Outstanding work"]
[null,"EXPLANATION ","Unique task code","This column includes all tasks that should be completed to satisfactorily quality assure a model. Some tasks are specific to Excel-based models. If your model is not Excel-based we recommend that you record relevant comments and evidence where possible for the software used or to mark it as N/A if that is not possible.","This could include:\n - a link to the template;\n - a link to guidance;\n - an explanation of how to perform the check.","Name of the person who performed the check","Rating of the model performance against the QA requirement. Follow the guidance in the column to the right.","Assessment of performance (from 1-5) against the QA task. The general guideline for the rating is as follows:\n\n1) delivered, fully in place and all risks managed\n2) delivery almost complete and any residual risks in hand\n3) mostly delivered, some outstanding risks or issues\n4) some elements delivered, but some work outstanding or risks not fully handled\n5) some elements are not delivered, or substantial risks remain\n","Explanation of the rating assigned.\n\nHow is the model performing against that QA requirement? Are there any issues? \n\nIf the QA requirement is not applicable to the model explain here why.","Link to documentation confirming the status of the QA on a given task - for higher ratings (1-2) evidence should to be available.","This includes all outstanding actions required to achieve the highest rating. You should label these actions as \"Essential\", \"Should have\" and \"Nice to have\" according to their importance."]
[null,"EXAMPLE","D1","Scope Document\nDoes a clear and comprehensive project scope exist with evidence of key stakeholder involvement and sign off?  This may be part of the current model report template, a separate scope document, or for small spreadsheet models embedded into the spreadsheet itself.","Please open the Model Report template for guidance on documentation","John Smith (senior engineer, BEIS)","3) Some issues","1) An up to date scope exists including all information contained in the template, and model boundaries and limitations are well specified. (note - this scope can either be a standalone document or covered within other documentation)\n2) Near-complete scope exists with the information contained in the template, and  model boundaries and limitations are mostly specified.\n3) Scope exists with some information from the template, but some elements are now out of date or model boundaries and limitations are partially specified.\n4) Scope exists but was never reviewed and it doesn't match any longer with the model, or model boundaries and limitations are not well specified.\n5) Scope missing.","The scope document exists but was not updated or reviewed. The model has been subject to several updates from the  time the scope document was written so limitations and key exclusions need to be reviewed."," [link to the document]","Must\n\nShould\nThe [model team] should meet with the relevant stakeholder group to check whether the scope document is still relevant, compare it with the template provided and update it if necessary. \n\nCould"]

{"sheet":"Structure","index":1,"n_rows":27,"n_cols":1,"rows_kept":27,"cut":false,"hidden":false,"merged":[[18,0,25,0],[26,0,34,0],[10,0,17,0]]}
[]
[{"$f":"IF(SecurityClassification=\"OFFICIAL\",\"\",SecurityClassification & \" - \")  & DocumentName & \" QA log - version \" & VersionNumber","v":" -  QA log - version "}]
[]
[]
[]
[]
[]
[]
[]
[]
["WHY ARE WE DOING THIS"]
[]
[]
[]
[]
[]
[]
[]
["QA REQUIREMENTS"]
[]
[]
[]
[]
[]
[]
[]
["CHECKLIST"]

{"sheet":"Summary Page","index":2,"n_rows":36,"n_cols":4,"rows_kept":36,"cut":false,"hidden":false,"merged":[]}
[]
[{"$f":"IF(SecurityClassification=\"OFFICIAL\",\"\",SecurityClassification & \" - \")  & DocumentName & \" QA log - version \" & VersionNumber","v":" -  QA log - version "}]
[]
[]
[null,null,null,"Guidance"]
[null,"Security Classification",null,"Consider whether the log may contain sensitive information and select the appropriate security classification"]
[]
[null,"Model Name",null,"Name of the model, not just the acronym"]
[]
[null,"Model Version",null,"Version of the model on which the QA log is based on"]
[]
[null,"Model analyst/developer",null,"Main analyst / developer of the model"]
[]
[null,"QA review performed by",null,"Who completed given version of the log"]
[]
[null,"Date of QA review",null,"Date when the QA log has been completed and shared with BEIS QA contact"]
[]
[null,"Reason for review",null,"- \"QA log created\" if it is the first log completed for the model\n- \"QA update\" if the log has been updated in some places after some model developments. No comprehensive review carried out\n- \"QA review (non MIT)\" if a comprehensive review has been carried out either by the developer or by someone else but not by MIT\n- \"MIT formal review\" if the log has been completed after BEIS Modelling Integrity Team (MIT) formal review"]
[]
[]
[null,"REVIEW BELOW "]
[]
[null,"Worksheet Descriptions - 5 main QA sections","Worksheet Name","Why this matters"]
[null,null,"Documentation","A model may be developed over a number of years by a variety of developers - therefore a process relying on a model is at risk from lack of or poor documentation.\nDocumentation is critical for allowing the transfer of knowledge from developers to users, auditors and future developers. No or limited documentation / commentary to explain the analysis or model may result in inappropriate use or inappropriate adoption for a new problem. It is prudent to have informative user guides and technical documentation, and to ensure that a number of staff are sufficiently well versed in a model to cope with any sudden departures or absence of key staff. Further to this, documentation should provide the relevant SRO with evidence of model requirements, governance approval and stakeholder engagement through the process. Finally, having an agreed scope & specification ensures that modellers, customers and other stakeholders understands what the model will and will not do (\"acceptability criteria\") given time and resource constraints and unnecessary modelling is reduced or stopped.\n\nIt is important that effort spent on documentation is proportionate, and there are various ways in which good documentation can be achieved including providing comments on the code in the model itself.  It isn’t always necessary to have a report entitled ‘Documentation’."]
[null,null,"Structure & Clarity","A well built spreadsheet is clearer to follow for the modeller and the end-user.  This means that errors are less likely to occur and are easier to spot when they do.  This reduces workload and overhead at all later stages, which reduces cost to the department."]
[null,null,"Verification","Verification is the process of checking that a model meets specifications and that it fulfils its intended purpose. Verification is used to check that \"the equations are being solved right\". It checks that the equations have been implemented correctly, and is generally thought of as being more objective than validation. "]
[null,null,"Validation"," Validation is used to check that \"the right equations are being solved\", i.e. it checks that the correct model is being used. This is done by implementing a number of tests."]
[null,null,"Data & Assumptions","Understanding the quality of data used as an input to a model will ensure that there is a better understanding of the output.  No matter how well designed and built a model is, outputs will be heavily dependent upon input values.  Sometimes, lower quality data is unavoidable, which is why determining the impact / uncertainty of this to the total model output (with sensitivity analysis etc..) is critical."]
[]
[]
[null,"Worksheet Descriptions - Information on the model QA","Model Score","Score for the model based on BEIS Model Scoring System"]
[null,null,"QA plan","Sheet to list QA activities that will be carried out on the model in order to reach a QA score of 90% or more. "]
[null,null,"QA history","A log of the evolution of the QA of the model"]
[null,null,"Issues Log","Log of errors with sheet and cell references"]
[null,null,"Version Log","Version control of the QA log itself"]
[null,null,"Lookups","Dimensions used in the data lists and weights used to calculate the model score"]

{"sheet":"Documentation","index":3,"n_rows":17,"n_cols":10,"rows_kept":17,"cut":false,"hidden":false,"merged":[]}
[]
[{"$f":"IF(SecurityClassification=\"OFFICIAL\",\"\",SecurityClassification & \" - \")  & DocumentName & \" QA log - version \" & VersionNumber","v":" -  QA log - version "}]
[]
[null,"QA should be proportionate to the model complexity and importance. The tasks described below are appropriate for complex, business critical models and should be interpreted flexibly according to the model specific characteristics and status. Contact the Modelling Integrity Team if you have any doubts"]
[null,"DECCModellingintegrityTeam@beis.gov.uk"]
[null,"(Note- changes in blue were made in June 2016, so these may be different from previous versions of this log you have seen)"]
[null,"Task Code","QA requirement","Guidance and links","Performed by","Rating","Rating guidelines","Current status (issues found)","Evidence (when applicable)","Outstanding work"]
[null,"Example","Scope Document\nDoes a clear and comprehensive project scope exist with evidence of key stakeholder involvement and sign off?  This may be part of the current model report template, a separate scope document, or for small spreadsheet models embedded into the spreadsheet itself.","Please open the Model Report file for guidance on documentation","John Smith (senior engineer, BEIS)","3) Some issues","1) An up to date scope exists including all information contained in the template, and model boundaries and limitations are well specified. (note - this scope can either be a standalone document or covered within other documentation)\n2) Near-complete scope exists with the information contained in the template, and  model boundaries and limitations are mostly specified.\n3) Scope exists with some information from the template, but some elements are now out of date or model boundaries and limitations are partially specified.\n4) Scope exists but was never reviewed and it doesn't match any longer with the model, or model boundaries and limitations are not well specified.\n5) Scope missing.","The scope document exists but was not updated or reviewed. The model has been subject to several updates from the  time the scope document was written so limitations and key exclusions need to be reviewed."," [link to the document]","Must\n\nShould\nThe [model team] should meet with the relevant stakeholder group to check whether the scope document is still relevant, compare it with the template provided and update it if necessary. \n\nCould"]
[null,"D1","Scope Document\nDoes a clear and comprehensive project scope exist with evidence of key stakeholder involvement and sign off?  This may be part of the current model report template, a separate scope document, or for small spreadsheet models embedded into the spreadsheet itself.","Please open the Model Report file for guidance on documentation",null,null,"1) An up to date scope exists including all information contained in the Model Report template, and model boundaries and limitations are well specified. (note - this scope can either be a standalone document or covered within other documentation)\n2) Near-complete scope exists with the information contained in the template, and  model boundaries and limitations are mostly specified.\n3) Scope exists with some information from the template, but some elements are now out of date or model boundaries and limitations are partially specified.\n4) Scope exists but was never reviewed and it doesn't match any longer with the model, or model boundaries and limitations are not well specified.\n5) Scope missing.",null,null,"Must\n\nShould\n\nCould"]
[null,"D2","Spec Document\nHave model specifications, defining precisely what the model is required to do, been completed and signed off?  This may be part of the current model report template, a separate spec document, or for small spreadsheet models embedded into the spreadsheet itself.\n\n","Please open the Model Report file for guidance on documentation",null,null,"1) An up to date specification exists including all information contained in the Model Report template, detailing all requirements of the model.  (note - this specification can either be a standalone document or covered within other documentation)\n2) Near-complete specification exists with the information contained in the template, detailing most requirements of the model.\n3) Specification exists with some information from the template, or the model requirements are partially specified.\n4) Specification exists but was never reviewed and it doesn't match any longer with the model, or model requirements are not well determined.\n5) Specification missing.",null,null,"Must\n\nShould\n\nCould"]
[null,"D3","User guide\nIs the user documentation sufficiently clear and able to support independent use of the model for a new model user who needs to run/operate the model and view outputs? (user guide could be within or outside of the model)","Please open the Model Report file for guidance on documentation",null,null,"1) Comprehensive user guide (or equivalent documentation within the model) enables model use, is easily found and up to date\n2) User guide available and enables model use but some information is missing or it is not up to date\n3) User guide not available, but running the model is easy and intuitive  \n4) User guide not available but a new user could understand key model functionalities relatively quickly\n5) No documentation available to support the user who cannot understand how to run the model in a relatively short time period.",null,null,"Must\n\nShould\n\nCould"]
[null,"D4","Technical guide\nIs technical documentation sufficiently clear and able to support independent maintenance and any future development of the model? This may be part of the current model report template or a separate technical document and may reference comments embedded within the model.","Please open the Model Report file for guidance on documentation",null,null,"1) Comprehensive and informative technical guide available and up to date \n2) Technical guide available but some information is missing or it is not up to date.\n3) Technical guide available but most information is missing or it is not up to date.\n4) Technical guide not available, but some technical info can be found within the model\n5) No technical documentation either in external document or within the model available to help users with future model adaptations.",null,null,"Must\n\nShould\n\nCould"]
[null,"D5","Intranet Page within BEIS\nAre up to date and clear documents accessible via BEIS's intranet? Even if the model contains sensitive information basic details should be shared via the intranet","For a 3rd-party model, you could either talk to your BEIS contact about this, or just mark it as n/a",null,null,"1) BEIS intranet page developed comprehensively in the standard template or equivalent.\n2) BEIS intranet page available with some information missing or out of date.\n3) BEIS intranet page available, but a lot of information is missing or out of date.\n4) The model's BEIS intranet page is missing, but it is possible to find some information about the model on the intranet.\n5) No information about the model is available on the intranet.",null,null,"Must\n\nShould\n\nCould"]
[null,"D6","Version Control\nDoes an up to date version control log exist? This may be embedded in the model.\nIs it clear which version of the model one is working on and what has changed since previous versions?\n","Version log available in the 'Logs' sheet of the Excel Template",null,null,"1) A version log exists and follows a clear and logical versioning system. Detailed information of the changes performed and their impacts are included.\n2) Version log and labelling exist and are largely complete but there are a few gaps or inconsistencies.\n3) A version log and labelling system exists but not all the versions were logged or the descriptions of the changes performed are limited\n4) No version log, but a version labelling makes it clear which model version is used\n5) No version log and no version labelling\n",null,null,"Must\n\nShould\n\nCould"]
[null,"D7","Governance and responsibilities\nHave responsibilities and accountabilities been assigned with sign off process completed? \n\nHave all relevant model / QA risks been identified, logged and escalated as necessary and with appropriate mitigation actions put in place? \n\nAre the relevant file organised in a sensible folder structure in a clear place? Are they accessible to the people who need them?","Sign-off must be recorded in the Model Report Template\nA risk log template can be found in the BEIS Excel template",null,null,"1) Clear evidence of accountabilities, risk management and signoff. Folder structure is simple and easy to understand with all relevant files saved in a clear place\n2) Near complete evidence of accountabilities, risk management and signoff. Good folder structure\n3) Some evidence of accountabilities, risk management and signoff. Some issues regarding folder structure\n4) Little evidence of accountabilities, risk management and signoff. Folder structure is confusing\n5) Insufficient evidence of accountabilities, risk management and signoff. It is difficult for new users to find the relevant files",null,null,"Must\n\nShould\n\nCould"]
[null,"D8","Model classification\nHas the model been assigned a security classification? \n\nIf the model uses sensitive data sources, are they clearly labelled? Is the access to sensitive information restricted to only the people that need to work with it?","For a 3rd-party model, you could either talk to your BEIS contact about this, or just mark it as n/a",null,null,"1) The model doesn't contain sensitive data sources and information OR data protection/classification procedures are correctly followed, security classification indicated for the whole model and for the relevant data sources.\n2) The model contains sensitive info and mostly complete data protection/classification procedures are in place\n3) The model contains sensitive info but only some data protection/classification procedures for model / data are in place. \n4) The model contains sensitive info and there are significant concerns about  data protection/classification procedures.\n5) No evidence that classifications have been considered or labels are in place even if the model contains sensitive data",null,null,"Must\n\nShould\n\nCould"]
[null,"D9","QA planning and resourcing\n\nHas an appropriate QA plan been implemented with adequate consideration of resource quantity and skills required? Does it give dates when each task will be complete, and do the plans give enough to reach the target score (90% for business-critical models, and 85% for other models)?\n\nDoes the plan include a list of specific checks that need to be done on the model before outputs are shared?\n\n(if applicable) is there evidence of QA processes carried out to date? Have the verification and validation activities been properly logged and commented? Have the recommendations highlighted in previous QA exercises (e.g. in the issues log) been implemented? ","Your QA plan should be stored within the 'QA plan' sheet in this file.",null,null,"1) A comprehensive QA plan is embedded within this QA log. QA reviews have been (or are planned to be) periodically carried out and documented. Time and resources have been planned to address QA recommendations, and the actions listed will get the model to the target score.\n2) Mostly complete evidence of QA plan and planned/previous QA work., and the actions listed will get the model to the target score.\n3) Some complete evidence of QA plan and previous QA work, but future actions are not enough to reach the target score.\n4) Little evidence of QA plan and previous QA work.\n5) No evidence of QA plan and previous QA work.",null,null,"Must\n\nShould\n\nCould"]

{"sheet":"Structure & Clarity","index":4,"n_rows":16,"n_cols":10,"rows_kept":16,"cut":false,"hidden":false,"merged":[]}
[]
[{"$f":"IF(SecurityClassification=\"OFFICIAL\",\"\",SecurityClassification & \" - \")  & DocumentName & \" QA log - version \" & VersionNumber","v":" -  QA log - version "}]
[]
[null,"QA should be proportionate to the model complexity and importance. The tasks described below are appropriate for complex, business critical models and should be interpreted flexibly according to the model specific characteristics and status. Contact the Modelling Integrity Team if you have any doubts"]
[null,"DECCModellingintegrityTeam@beis.gov.uk"]
[null,"(Note- changes in blue were made in June 2016, so these may be different from previous versions of this log you have seen)"]
[null,"Task Code","QA requirement","Guidance and links","Performed by","Rating","Rating guidelines","Current status (issues found)","Evidence (when applicable)","Outstanding work"]
[null,"Example","Model structure\nIs a map of the model available?\n\nIs there a clear distinction between inputs, calculations and outputs and no unnecessary circularity across worksheets (i.e. if worksheet A refers to worksheet B then worksheet B should NOT refer to worksheet A)? \n\nIs the structure clear and, as far as possible, simple (ideally input sheets should affect only calculation sheets which should affect only output sheets)? \n\nDoes the model contain any data / structure /clutter which serves no apparent purpose?","Guidelines on best practice about structure can be found in the \"Overview\" sheet of the Excel Template\n\nAn example of good structure is the 2050 Calculator model (http://2050-calculator-tool-wiki.decc.gov.uk/pages/140)\n","John Smith (senior engineer, BEIS)","3) Some issues","1) A model map exists and has been used to identify possible issues with the structure. The model structure is clear, simple and free of unused data / formulae.\n2) The model has a clear and simple structure, but a map is not available. May include a small amount of unused data / formulae.\n3) Some issues with the structure: the number of interactions between sheets is quite high and no clear distinction between inputs, calculations and outputs. May include some unused data / formulae.\n4) Many issues about the structure: it is not possible to describe easily the calculation flow. Model could include significant amounts of unused data / formulae. \n5) Extremely complicated structure. It is not clear how the calculation flows (i.e. which sheets affect which and how inputs are transformed in outputs). Model could include substantial amounts of unused data / formulae. ","A model map exists  (both within the model and in high level documentation) but it is out of date. MIT have produced an influence matrix showing how the sheets interact (see 'Influence Matrix' tab in this QA log) - this shows several areas of circularity between tabs, e.g.' Sheet1' feeds into 'Sheet2' and vice versa. Tabs sometimes refer to other tabs that come after/to the right of them, meaning that the workbook doesn't read like a book.\nInputs, calculations and outputs are generally separated into different tabs and tab names have prefixes (Inp), (Calc), (Res) to indicate their predominant type (although they usually also include other types).\nThe structure is relatively clear, but there is quite a lot of redundancy due to different policy options being considered over time and not removing this functionality once the final option has been determined. Many of the policy scenarios available via drop-down boxes are no longer being considered and produce  incorrect results if selected.","See 'Influence Matrix' tab","Must\nClearly mark any tabs that do not currently function correctly \nRemove non-functioning scenarios from drop-down lists\n\nShould\nUse the MIT 'Redundancy tool' to identify redundant areas of the model, and remove these. (or review spreadsheet pros lists of unused data/calculations)\nReview the 'Influence Matrix' tab and consider whether any areas of circularity can be improved or whether any of the structure can be simplified, so that it reads like a book. Re-run this after removing any redundancy in the model\n\nCould"]
[null,"S1","Model structure\nIs a map of the model available?\n\nIs there a clear distinction between inputs, calculations and outputs and no unnecessary circularity across worksheets (i.e. if worksheet A refers to worksheet B then worksheet B should NOT refer to worksheet A)? \n\nIs the structure clear and, as far as possible, simple (ideally input sheets should affect only calculation sheets which should affect only output sheets)? \n\nDoes the model contain any data / structure /clutter which serves no apparent purpose?","Guidelines on best practice about structure can be found in the \"Overview\" sheet of the Excel Template\n\nAn example of good structure is the 2050 Calculator model (http://2050-calculator-tool-wiki.decc.gov.uk/pages/140)\n",null,null,"1) A model map exists and has been used to identify possible issues with the structure. The model structure is clear, simple and free of unused data / formulae.\n2) The model has a clear and simple structure, but a map is not available. May include a small amount of unused data / formulae.\n3) Some issues with the structure: the number of interactions between sheets is quite high and no clear distinction between inputs, calculations and outputs. May include some unused data / formulae.\n4) Many issues about the structure: it is not possible to describe easily the calculation flow. Model could include significant amounts of unused data / formulae. \n5) Extremely complicated structure. It is not clear how the calculation flows (i.e. which sheets affect which and how inputs are transformed in outputs). Model could include substantial amounts of unused data / formulae. ",null,null,"Must\n\nShould\n\nCould"]
[null,"S2","Worksheet structure \nAre calculation flows within worksheet logical and easy to understand? \n\nIs the structure consistent across worksheets? Do similar worksheets have similar structures? Are similar tables laid out in similar way? \n\nIs the model free from anomalous calculation/label/text cells (No undocumented inconsistencies with formulae)?",null,null,null,"1) All worksheets have a clean layout and similar worksheets present similar structure\n2) All high priority calculations have a clean layout. Other parts of the model would be clear to someone after reading documentation or basic training.\n3) Most sheets have a good layout, but some are quite complicated and present several issues (e.g. unnecessary big sheets, not aligned tables, confused structure)\n4) Many sheets present issues\n5) It is difficult to understand many sheets because the layout is very poor or the structure is confused",null,null,"Must\n\nShould\n\nCould"]
[null,"S3","Labels, Units & Conversions\nAre titles, labels and units present, logical and accurate? These checks should include charts, tables and text.\n\nIs a table used to perform unit conversions?\n\nIs rounding performed in a correct way? Are rounded numbers and original numbers used in the correct places?","Check that every input, table and output are labelled and with units. The sheet \"Unit Conversion\" in the Excel Template shows how to correctly perform unit conversions.",null,null,"1) Units and labels always clearly expressed. Unit conversion is performed using conversion tables so that no hardcoded values are used. Rounding is performed correctly\n2) Units and labels always clearly expressed. Conversion tables are available but in few cases conversions are performed using hardcoded values\n3) Units and labels are available for all the main calculations but missing in less important model sections. Or unit conversions often performed using hardcoded values\n4) Units or labels sometimes missing even in main calculations, or conversions mostly performed with hardcoded values\n5) Units or labels often missing, or conversions performed with hardcoded values",null,null,"Must\n\nShould\n\nCould"]
[null,"S4","Formatting\nDoes formatting clearly distinguish inputs, outputs and calculations, and aid understanding?  Is it clear through formatting when a different formula is used within the same section?\n\nAre the parts of the models currently not used or under development easily recognisable through formatting? \n\nIs it clear through formatting when data is taken from different sources?","The 2050 Calculator provides good examples of formatting:\nhttps://www.gov.uk/government/publications/2050-pathways-calculator-with-costs ",null,null,"1) Formatting is used consistently across the workbook. A legend clearly explains the formatting used\n2) Formatting is used on all main sheets in a consistent and documented way; there are some minor areas of the workbook that would benefit from formatting\n3) Formatting is used on most sheets and is consistent but is not explained and does not cover all aspects of the workbook\n4) Some formatting is used but not in every sheet or it is not always consistent\n5) Formatting is barely used or is contradictory within the model.",null,null,"Must\n\nShould\n\nCould"]
[null,"S5","Workbook Comments\nIs model sufficiently annotated? Are comments up to date?\n\nAre all the data sources explicitly documented in the model? \n\nAre complex formulae sufficiently explained? \n\nIf formulae change midway through an array, row or column are there comments to state this has happened and why?\n\nAre there short descriptions of the content and logic of every sheet?",null,null,null,"1) Comments and explanations are embedded across the workbook so that it is very easy to understand formulae and methodology. Data sources (and links to them) are always present.\n2) The majority of the model (including the high priority formulae) has comments embedded and these are clear but there are some cases where comments would be helpful. Data sources (and links to them) are always present.\n3) The main parts of the model are commented, but some further explanations would be helpful. Not all the sources are linked within the model\n4) There are some comments but these are not clear and some are out of date. It is not possible to fully understand the model without talking with the modellers\n5) No or very rare comments. It is very difficult to understand formulae without talking with the modellers",null,null,"Must\n\nShould\n\nCould"]
[null,"S6","Code Comments\nIs the VBA code sufficiently and appropriately commented? \n\nIs there a description of what each macro does?\n\nIf the model doesn't contain code, you can mark this QA task as N/A",null,null,null,"1) All the significant lines of the VBA code are commented. It is clearly and exhaustively explained how the macros work and how they are used in the model\n2)The most important parts of the code are sufficiently commented and it is immediately clear how the macros work but some minor sections would benefits from more comments\n3) Comments are present most of the time but there are gaps and some further explanations would aid understanding\n4) There are some comments but the code is not easy to understand without investigation.\n5) There are very few comments or they are not properly explained. It is not at all easy to understand what the macros do or how they are used in the model. \nN/A) No code (e.g. the model is just in Excel with no VBA)",null,null,"Must\n\nShould\n\nCould"]
[null,"S7","Formula clarity & robustness\nAre formulae easily understandable and robust?\n\nIs the model free from hardcoded values? \n\nAre array formulas justified? Can an alternative method be used? \n\nAre merged cells avoided for inputs, calculations and outputs?","Formula clarity could be enhanced using  the command \"alt+enter\"  to have the right spacing. Hardcoded values must be avoided as they are difficult to understand and could lead to potential issues.",null,null,"1) Formulae are, as far as possible, easily understandable. No potential issues due to lack of formulae robustness (e.g. if changes are introduced, the model goes on working fine). No hardcoded values\n2) A robust approach is used in most of the formulae (particularly those which are high-risk) Some minor issues, like a few hardcoded values\n3) Some formulae could be made more robust (e.g. by using tables or named ranges or removing the hardcoded values). \n4) In many cases the formulae used are not the best/most robust ones. Changes in the model, such as the introduction of new functionalities or the change of the default setting would likely lead to issues. Hardcoded values are used extensively.\n5) Most formulae are not clear or robust, large use of hardcoded values",null,null,"Must\n\nShould\n\nCould"]
[null,"S8","Named range management\nAre names used to aid user understanding and simplify formulae? \n\nDo names follow a common convention? \n\nIs the workbook free from corrupted or wrong ranged names?\n\nMark this as N/A if the use of named ranges wouldn't make the model more robust","Named ranges (or tables) should be used extensively in the model as they are more robust and avoid potential issues. They must be used in macros, as they are not updated in the code when something is changed in the model\n\nCheck the Name Manager (ctrl+F3) to ensure there are no corrupted names, and to check that external ranged names are not present unless it is absolutely necessary. Also check that the names are meaningful and follow a common convention. ",null,null,"1) Named ranges are extensively and correctly used. A common naming convention helps understand what each named range is. No corrupted or wrong names\n2) Named ranges are mostly used with a clear naming convention but they could be used more to aid understanding. Limited corrupted or wrong names.\n3) There are some named ranges in key areas and the names are correctly used. However, no naming convention and/or a few corrupted or redundant names.\n4)There are some named ranges in the model but they are not consistently managed and there are many corrupted, unclear or redundant names.\n5) There are no or very few named ranges in the model and those present do not add clarity to the model.\nN/A The model would not benefit from the use of named ranges",null,null,"Must\n\nShould\n\nCould"]

{"sheet":"Verification","index":5,"n_rows":13,"n_cols":10,"rows_kept":13,"cut":false,"hidden":false,"merged":[]}
[]
[{"$f":"IF(SecurityClassification=\"OFFICIAL\",\"\",SecurityClassification & \" - \")  & DocumentName & \" QA log - version \" & VersionNumber","v":" -  QA log - version "}]
[]
[null,"QA should be proportionate to the model complexity and importance. The tasks described below are appropriate for complex, business critical models and should be interpreted flexibly according to the model specific characteristics and status. Contact the Modelling Integrity Team if you have any doubts"]
[null,"DECCModellingintegrityTeam@beis.gov.uk"]
[null,"(Note- changes in blue were made in June 2016, so these may be different from previous versions of this log you have seen)"]
[null,"Task Code","QA requirement","Guidance and links","Performed by","Rating","Rating guidelines","Current status (issues found)","Evidence (when applicable)","Outstanding work"]
[null,"Example","Formula correctness\nAre all formulae correctly implemented? Have they all been checked? Have the high priority sections of the model been identified?\n\nDo all formulae refer to the right cell? Have formulae been copied down and across as far as they should be?\nIs the data being pulled into the calculation modules correctly?\n\nDo numbers apply to the correct time period (e.g. the middle of the month/year versus the beginning/end)? Is it consistent throughout the model? Are financial year and calendar year data managed correctly? Are discount rates (nominal/real) applied correctly?","\nDetailed inspection: check formula functionality against row & column titles - if not clear then there are implicit or undocumented assumptions  and you should clarify what the formula is doing.  \n\nCheck the Name Manager (ctrl+F3) to ensure all the range names are correct and cover the full range.","John Smith (senior engineer, BEIS)","2) Good","1) All formulae and names have been checked after the last model development and no issues have been found or they have all been fixed.\n2) All the high priority formulae and names have been checked after the last model development and the important issues have been fixed. There is a good QA process in place to check the remaining formulae. The few minor outstanding issues are fully documented and have no impact on the results.\n3) Most of the formulae and the names have been checked recently. Some issues are outstanding. \n4) Most of the formulae and names have been checked recently. Some formulae or names are not correct and could have an impact on the model outputs.\n5) Many formulae or names are not correct and they have a significant impact on the model outputs, OR very few formulae and names have been checked.","The model has undergone in June 2013 an external review which has audited model formulae. The modelling team have implemented corrections and suggestions for improvement  identified as part of the external review. Few outstanding issues, which don't affect the outputs are in the Issues Log","[Links to external audit reports]\n\nSee Issues Log","Must\n\nShould\nReview all issues identified in the Issues Log and correct as appropriate.\n\nCould"]
[null,"Ve1","Formula correctness\nAre all formulae correctly implemented? Have they all been checked? Have the high priority sections of the model been identified?\n\nDo all formulae refer to the right cell? Have formulae been copied down and across as far as they should be?\nIs the data being pulled into the calculation modules correctly?\n\nDo numbers apply to the correct time period (e.g. the middle of the month/year versus the beginning/end)? Is it consistent throughout the model? Are financial year and calendar year data managed correctly? Are discount rates (nominal/real) applied correctly?","\nDetailed inspection: check formula functionality against row & column titles - if not clear then there are implicit or undocumented assumptions  and you should clarify what the formula is doing.  \n\nCheck the Name Manager (ctrl+F3) to ensure all the range names are correct and cover the full range.",null,null,"1) All formulae and names have been checked after the last model development and no issues have been found or they have all been fixed.\n2) All the high priority formulae and names have been checked after the last model development and the important issues have been fixed. There is a good QA process in place to check the remaining formulae. The few minor outstanding issues are fully documented and have no impact on the results.\n3) Most of the formulae and the names have been checked recently. Some issues are outstanding. \n4) Most of the formulae and names have been checked recently. Some formulae or names are not correct and could have an impact on the model outputs.\n5) Many formulae or names are not correct and they have a significant impact on the model outputs, OR very few formulae and names have been checked.",null,null,"Must\n\nShould\n\nCould"]
[null,"Ve2","Usability Testing\nCan a new user easily operate the model and view outputs?  \n\nDo hyperlinks, macros and buttons work? Is routine operation of the model smooth and free of bugs? \n\nDoes the model open in an acceptable amount of time? Is the runtime of the model appropriate for the demand placed on the model and the complexity of what is being modelled?\n\nAre there #REF, #DIV/0, #VALUE errors in the model?\n\nHave the appropriate control been put in place to avoid invalid data/entries (e.g. not sensible inputs)? Have the relevant cells been protected to avoid the user introducing unwanted changes? If relevant, have table titles been locked from editing?","Check that all the user options can be selected without generating errors or unexpected results. Routines work without runtime errors. There are no buttons with dead links\n\nUse search function for \"#\" - refer to V&V standard for potential solutions (remove faults, or document them where unavoidable). \n\nCheck all the cells where data validation is used or should be used to protect the appropriate cells. Guidance on how to protect cells in Excel: http://office.microsoft.com/en-us/excel-help/lock-cells-in-a-worksheet-HA010342979.aspx ",null,null,"1) The model is free of bugs and all the functionalities work correctly. If parts of the model are a work in progress, they are clearly labelled and formatted. When settings or scenarios are changed no issues occur. The runtime is appropriate. All the relevant cell protection procedures have been implemented. \n2) The model is free of bugs and all the functionalities work correctly. There are some #REF, #DIV/0 and #VALUE errors which don't have any impact on the model and have been documented and explained. \n3) The main functionalities work correctly. Some issues if the non-default scenarios/settings are selected, but they have been documented. Some protection procedures implemented but a few issues occur (data list is not up to date etc.)\n4) Routine operation works correctly, but change of scenarios/settings leads to errors. Issues with model runtime. Limited protection procedures and a moderate change of some major errors creeping in.\n5) There are bugs during routine operation, no cell protection implemented OR usability has not been checked. ",null,null,"Must\n\nShould\n\nCould"]
[null,"Ve3","Code correctness\nDoes the code function as intended without error, and produce the intended results? \n\nAre hard-coded references to cells used only when absolutely necessary? If they are used, are they referring to correct values?\n\nHave all variables been declared (helps ensure variables are spelt correctly/memory is used appropriately)? Is appropriate error handling built in?\n\nIf the model doesn't contain code mark this QA task as N/A","Review the logic of the macro code (subroutines & macros match stated purpose and function. Check each line in the code). “Step through” the code to visually inspect its operation (ensure that no unintended behaviour is observed). Hard-coded references are NOT updated in vba code when rows/columns are added, so check any hard-coded references are correct, and commented within the body of the code. ",null,null,"1) All the code (e.g VBA) has been checked after the last model development and no issues have been found or they have all been fixed. \n2) All the high priority parts of the code have been checked after the last model development and the important issues have been sorted out. The few minor outstanding issues are fully documented and have no impact on the results.\n3) Most of the code has been checked recently. Some issues are outstanding. \n4) Most of the code has been checked recently. Part of the code is not correct and could have an impact on the model outputs.\n5) Part of the code is not correct and this has a significant impact on the model outputs, OR very little of the code has been checked.\nN/A) No code present in the model. (e.g. the model is just in Excel with no VBA)",null,null,"Must\n\nShould\n\nCould"]
[null,"Ve4","External links\nAre links to external documents used only when absolutely necessary? Are they properly documented? \n\nIf external links are used, do they pull in the correct, up to date data? Can the external data be 'refreshed'?","Open file on a different machine to original to ensure no undocumented error messages when opening file\n\nCheck the external links to ensure the most up to date data are used and no errors occur",null,null,"1) No external links, or external links are used only where necessary and are properly documented. Data are up to date and no errors occur when updating these links.\n2) A few external links are not documented but they all work correctly, OR most external links have been checked\n3) Some external links are not documented, but work correctly, OR only some external links have been checked.\n4) External links are present in many formulae and make it difficult to run the model on a different machine; OR few external links checked\n5) Many external links are not documented and some errors occur, OR external links have not been checked.",null,null,"Must\n\nShould\n\nCould"]
[null,"Ve5","Autochecks & Regression Testing\nAre autochecks used to highlight correct functionality (e.g. a cell in Excel to highlight when all data is filled in), and are they implemented correctly? \n\nIs conditional formatting used to spot potential issues (e.g. negative values)?\n\nDoes the model make use of a reference set of input data / output data to test the model for unexpected outcomes after every significant development?\n\nAre there a number of quantitative scenarios that cover a wide range of the possible inputs which can be run on old and new versions of the model?","Sparklines could be used to perform quick auto-checks: https://support.office.com/en-us/article/Use-sparklines-to-show-data-trends-1474e169-008c-4783-926b-5c60e620f5ca\n\nCheck that autochecks are correctly implemented and produce expected results",null,null,"1) Auto-checking functionalities are present and correctly implemented wherever possible. A reference set of input/output data is used to test the model after every significant model development. \n2)  Auto-checking functionalities correctly implemented for high priority model sections, but a few could be added to improve model quality. Regression testing performed after every significant model development\n3) There are some autochecks, but more could be introduced to easily spot errors or inconsistencies, OR only some autochecks have been examined. Some regression testing performed but not consistently\n4)  Very few autochecks implemented OR few autochecks examined. Output changes have been sometimes checked after significant model development, but not in a consistent and automatic way\n5) No autochecks, OR these have not been examined. No regression testing",null,null,"Must\n\nShould\n\nCould"]

{"sheet":"Validation","index":6,"n_rows":13,"n_cols":10,"rows_kept":13,"cut":false,"hidden":false,"merged":[]}
[]
[{"$f":"IF(SecurityClassification=\"OFFICIAL\",\"\",SecurityClassification & \" - \")  & DocumentName & \" QA log - version \" & VersionNumber","v":" -  QA log - version "}]
[]
[null,"QA should be proportionate to the model complexity and importance. The tasks described below are appropriate for complex, business critical models and should be interpreted flexibly according to the model specific characteristics and status. Contact the Modelling Integrity Team if you have any doubts"]
[null,"DECCModellingintegrityTeam@beis.gov.uk"]
[null,"(Note- changes in blue were made in June 2016, so these may be different from previous versions of this log you have seen)"]
[null,"Task Code","QA requirement","Guidance and links","Performed by","Rating","Rating guidelines","Current status (issues found)","Evidence (when applicable)","Outstanding work"]
[null,"Example","Methodology correctness\nIs the methodology used sensible and fit for purpose? (with \"purpose\" defined in the model specification doc)\n\nWas the model methodology reviewed and agreed with relevant stakeholders? \n\nDoes the model produce logical outputs (i.e. what you expect, that is, values in the direction of travel and of the right magnitude) when subject to sense / reality checks )? Have model outputs been sense / reality checked and agreed with relevant stakeholders?","Review the model logic eg. checking the actual flow of data through the model against a stylised example of how data is perceived to be flowing through the model.\n\nCheck if the outputs of different scenarios (high, medium, low) are sensible.\n\nAn example of good documentation of a model methodology is this 2050 Calculator webpage: http://2050-calculator-tool-wiki.decc.gov.uk/pages/142","John Smith (senior engineer, BEIS)","1) Excellent","1) The methodology has been agreed with relevant stakeholders and reviewed by experts. The model produces logical outputs\n2) The methodology is correct but only some relevant stakeholders have reviewed it. The model produces logical outputs\n3) Both methodology and outputs are logical, but have not been reviewed by relevant stakeholders, OR only parts of the methodology have been reviewed since a redevelopment\n4) Some results are counterintuitive and not all the methodology is sensible, OR very few parts of the methodology have been reviewed\n5) Not all the outputs are logical or the methodology is not always sensible and fit for purpose, OR the methodology needs review but this has not yet happened","The methodology is correct and exhaustively explained in the documentation; the model matches the spec document.\nSome sense checks have been performed and in all the cases the outputs seemed sensible. Greater expertise about the policy effects would be required to interpret more detailed tests, but the model has been used and checked by stakeholders several times.","[link to sense check testing documents]","Must\n\nShould\n\nCould\nCarry out additional tests to check model outputs"]
[null,"Va1","Methodology correctness\nIs the methodology used sensible and fit for purpose? (with \"purpose\" defined in the model specification)\n\nWas the model methodology reviewed and agreed with relevant stakeholders? \n\nDoes the model produce logical outputs (i.e. what you expect, that is, values in the direction of travel and of the right magnitude) when subject to sense / reality checks )? Have model outputs been sense / reality checked and agreed with relevant stakeholders?","Review the model logic eg. checking the actual flow of data through the model against a stylised example of how data is perceived to be flowing through the model.\n\nCheck if the outputs of different scenarios (high, medium, low) are sensible.\n\nAn example of good documentation of a model methodology is this 2050 Calculator webpage: http://2050-calculator-tool-wiki.decc.gov.uk/pages/142",null,null,"1) The methodology has been agreed with relevant stakeholders and reviewed by experts. The model produces logical outputs\n2) The methodology is correct but only some relevant stakeholders have reviewed it. The model produces logical outputs\n3) Both methodology and outputs are logical, but have not been reviewed by relevant stakeholders, OR only parts of the methodology have been reviewed since a redevelopment\n4) Some results are counterintuitive and not all the methodology is sensible, OR very few parts of the methodology have been reviewed\n5) Not all the outputs are logical or the methodology is not always sensible and fit for purpose, OR the methodology needs review but this has not yet happened",null,null,"Must\n\nShould\n\nCould"]
[null,"Va2","Comparison with historical data / backcasting / alternative models\nDoes the model match historical results when using historical input data, to within an agreed tolerance level?\nOr is there an alternative model to which the model can be compared?\n\nIf there are no historical data mark this as N/A","This could include also a cross-check of model outputs against an alternative set of data or information",null,null,"1) The model behaviour when historical data are used is sensible and the outputs are fairly close to the historical outputs OR the model has been compared against alternative data sources, with a clear explanation of any differences\n2) Most of the outputs match historical or alternative data with a clear explanation of any differences\n3) Historical outputs cannot be always reproduced by the model, but this behaviour is documented, OR only some of the feasible comparisons have been carried out.\n4) The historical values differ significantly from the model outputs and the explanation for this is somewhat insufficient, OR few comparisons with historical data performed\n5) The model cannot replicate the historical outputs and no explanation is available, OR no historical comparisons have been performed.\nN/A) No historical data available and no alternative data comparison is possible",null,null,"Must\n\nShould\n\nCould"]
[null,"Va3","Uncertainty testing\nHas data (especially poor quality data) been addressed via sensitivity testing and/or Monte Carlo analysis, and the results documented? Has uncertainty about the structure of the analysis (e.g. equations or logic used to combine the analysis inputs) been quantified through uncertainty analysis?\n\nHas sensitivity analysis been used to find out the most significant inputs and assumptions?\nWhere random sampling is used, is it as consistent as possible with real-world distributions?\n\nAre poor quality assumptions managed via sensitivity testing to determine the potential impact of an incorrect assumption?\n\nHave scenario combinations been determined and performed to check whether expected outputs are produced?\n\nIf the model just perform standards calculations and there is no uncertainty mark the task as N/A","A tool called \"@RISK\" can help you to perform Monte Carlo analysis \n\n",null,null,"1) Appropriate sensitivity testing has been performed on all the relevant data and assumptions. The model always responds logically to these tests and the potential impacts of a change in data or assumptions have been fully recorded. Scenario testing have been performed and the expected outputs have been obtained\n2) Appropriate sensitivity testing performed for all high risk data and assumptions. Results from these tests are logical and have been recorded\n3) Only some scenarios considered or testing not properly logged\n4) Few testing performed and results not logged and analysed\n5) No sensitivity testing performed, or results from testing are illogical and/or not documented. Scenario testing missing or very limited\nN/A) There is no uncertainty as the model simply performs standard calculations",null,null,"Must\n\nShould\n\nCould"]
[null,"Va4","Extreme values testing / model breaking\nDoes the model respond as expected to extreme values, zeroes, negative values, critical limits? \n\nIs it possible to 'break' the model or get implausible outcomes (e.g. percentages adding to more than 100%, people adding up to more than the population...)?","Check that entering extreme values to input data/ parameters does not break the model or result in unintended outcomes. Carry out checks by populating the model with dummy data: insert simple numbers like 0 or 1, where you know what the answer should be, and check for correctness",null,null,"1) Extreme values testing has been performed for all the relevant data. Errors due to extreme values have been identified and cell protection procedures have been implemented where appropriate so that only values in a certain range are acceptable inputs\n2) Extreme values testing has been performed on all high risk data and results are acceptable (no significant issues)\n3) Few extreme values tests have been performed but results are logical\n4) Not enough testing performed OR many issues (although not critical) found when performing extreme values tests\n5) No extreme values testing, or there are significant issues when testing extreme values\nN/A) Extreme values testing would not be helpful for the model",null,null,"Must\n\nShould\n\nCould"]
[null,"Va5","Re-performance testing\nCan you replicate model output by independently (e.g. offline) re-performing key calculations on sections of the model?","How the task is performed strongly depends on the nature of the model: in some cases it is feasible and worth creating an entire shadow model, in others just some calculations could be replicated in a different way. For example, test array formulas ({}) should be re-calculated by re-performing calculation manually",null,null,"1) All key model calculations have been reperformed and the same results have been achieved (or within acceptable tolerance level)\n2) Key model calculations have been reperformed and the same results have been achieved (or within acceptable tolerance level)\n3) Some model sections have been reperformed in a different way but the outputs differ somewhat.  However the reason  behind the difference is fully understood and documented\n4) There are some difference in the outputs and not all of them can be clearly explained\n5) No re-performance testing implemented even if it would be feasible and beneficial to check model outputs, or the testing produces unacceptably different values from the model's outputs, OR Re-performance testing has not been reviewed\nN/A) Re-performance testing not applicable",null,null,"Must\n\nShould\n\nCould"]

{"sheet":"Data & Assumptions","index":7,"n_rows":11,"n_cols":10,"rows_kept":11,"cut":false,"hidden":false,"merged":[]}
[]
[{"$f":"IF(SecurityClassification=\"OFFICIAL\",\"\",SecurityClassification & \" - \")  & DocumentName & \" QA log - version \" & VersionNumber","v":" -  QA log - version "}]
[]
[null,"QA should be proportionate to the model complexity and importance. The tasks described below are appropriate for complex, business critical models and should be interpreted flexibly according to the model specific characteristics and status. Contact the Modelling Integrity Team if you have any doubts"]
[null,"DECCModellingintegrityTeam@beis.gov.uk"]
[null,"(Note- changes in blue were made in June 2016, so these may be different from previous versions of this log you have seen)"]
[null,"Task Code","QA requirement","Guidance and links","Performed by","Rating","Rating guidelines","Current status (issues found)","Evidence (when applicable)","Outstanding work"]
[null,"Example","Data \n(Please note that a good assumptions log will cover the criteria for both DA1 and DA3 so you can almost certainly rate them both the same)\nDoes your assumptions log contain details of all model data? Has key data been identified and prioritised?\n\nHas appropriate data been used? \nAre the quality, characteristics, strengths and limitations of the data set fully understood and recorded? \nHave data quality, impact and risk been assessed? \n\nHave data inputs been agreed with relevant stakeholders and signed-off? Are all data up-to-date?","BEIS assumptions log","John Smith (senior engineer, BEIS)","3) Some issues","1) Assumptions log containing all the relevant information about data available. Data fit for purpose, their quality, impact and risk recorded and fully understood. Data signed-off and up-to-date\n2) Assumptions log containing all high risk data available; data fit for purpose, their quality, impact and risk recorded and fully understood. Data signed-off and up-to-date\n3) Data fit for purpose and Assumptions log available but not all data are recorded and/or some information is missing. Or assumptions log is missing but the majority of data is recorded and information is present within the model.\n4) Assumptions log missing, but some information is available (in the documentation or within the model). Quality of data not always assessed properly, sign off may be missing\n5) Assumptions log missing and very little information available. No documentation assessing data quality and limitations and no sign off. Not all data is fit for purpose","There is currently no formal sign off process for data  but it is agreed within teams. The data used has been widely consulted on and is transparent (available through the IA). The shortage of proven data for key variables has been made clear and significant consultation with key stakeholders on the numbers has taken place. Where data is more widely available the team have ensured crossover with other key BEIS models. Whilst information is available the data is not logged in such a way as to make it easy to track or the risks associated with individual data clear. Effective data management and assessment processes going forwards are crucial as more actual data becomes available to replace assumptions. ","[link to relevant documents]","Must\nBuild on the Inputs and Assumptions Log to give data used a RAG rating, link to evidence sources and make explicit the sign off procedure. \n\nShould\n\nCould\n"]
[null,"DA1","Data \n(Please note that a good assumptions log will cover the criteria for both DA1 and DA3 so you can almost certainly rate them both the same)\nDoes your assumptions log contain details of all model data? Has key data been identified and prioritised?\n\nHas appropriate data been used? \nAre the quality, characteristics, strengths and limitations of the data set fully understood and recorded? \nHave data quality, impact and risk been assessed? \n\nHave data inputs been agreed with relevant stakeholders and signed-off? Are all data up-to-date?","BEIS assumptions log",null,null,"1) Assumptions log containing all the relevant information about data available. Data fit for purpose, their quality, impact and risk recorded and fully understood. Data signed-off and up-to-date\n2) Assumptions log containing all high risk data available; data fit for purpose, their quality, impact and risk recorded and fully understood. Data signed-off and up-to-date\n3) Data fit for purpose and Assumptions log available but not all data are recorded and/or some information is missing. Or assumptions log is missing but the majority of data is recorded and information is present within the model.\n4) Assumptions log missing, but some information is available (in the documentation or within the model). Quality of data not always assessed properly, sign off may be missing\n5) Assumptions log missing and very little information available. No documentation assessing data quality and limitations and no sign off. Not all data is fit for purpose",null,null,"Must\n\nShould\n\nCould"]
[null,"DA2","Data transformation\nHas input data been checked against primary reference for potential errors in copying / pasting / transforming?\n\nIf required, have details on how the data have been imported/transformed or processed been recorded in the Assumptions Log? \n\nHave sources of data been documented in the Assumptions Log? ","Modellers should ensure that no errors are made when data is imported. In some occasions it may be appropriate to consider automatic way to import data, which should reduce the probability of errors",null,null,"1) Complete details available about where the data is taken and how it is transformed. Data matches source following transformation.\n2) Details on data transformation are available for all critical data and sources are included\n3) Not all the data transformations are clearly documented but no errors due to copying/pasting/transforming data\n4) Not clear description of data transformations, some sources are not included\n5) A proper description of how data is transformed is missing, sources are often not included. It is difficult to check whether there are errors due to copying/pasting/transforming data\nN/A) No transformation performed",null,null,"Must\n\nShould\n\nCould"]
[null,"DA3","Assumptions\n(Please note that a good assumptions log will cover the criteria for both DA1 and DA3 so you can almost certainly rate them both the same)\nDoes your assumptions log contain details of all model assumptions?  Have key assumptions been identified and prioritised?\n\nAre assumptions appropriate, applicable and logically coherent? \nAre any limitations/caveats adequately described in the Assumptions log? \nHave data quality, impact and risk been assessed? \nHave assumptions been agreed with relevant stakeholders and signed-off? Are assumptions up-to-date?","BEIS assumptions log",null,null,"1) Assumptions log containing all the relevant information about all the assumptions Assumptions fit for purpose, their quality, impact and risk recorded and fully understood. Assumptions signed-off and up-to-date\n2) Assumptions log containing all high risk assumptions; assumptions fit for purpose, their quality, impact and risk recorded and fully understood. Assumptions signed-off and up-to-date\n3) Assumptions fit for purpose and Assumptions log available but not all assumptions are recorded and/or some information is missing. Or assumptions log is missing but the majority of assumptions is recorded and information is present within the model.\n4) Assumptions log missing, but some information is available (in the documentation or within the model). Quality of assumptions not always assessed properly, sign off may be missing\n5) Assumptions log missing and very little information available. No documentation assessing assumptions quality and risk and no sign off. Not all assumptions are fit for purpose",null,null,"Must\n\nShould\n\nCould"]

{"sheet":"Model score","index":8,"n_rows":80,"n_cols":12,"rows_kept":80,"cut":false,"hidden":false,"merged":[[15,11,27,16],[8,11,13,16],[6,11,6,16]]}
[]
[{"$f":"IF(SecurityClassification=\"OFFICIAL\",\"\",SecurityClassification & \" - \")  & DocumentName & \" QA log - version \" & VersionNumber","v":" -  QA log - version "}]
[]
[null,"MODEL QA SCORE",null,null,null,null,null,null,null,null,"SCORE EXPLANATION"]
[]
[null,null,null,null,"Overall Score",null,null,null,null,null,null,"Score meaning"]
[null,null,null,null,{"$f":"SUMPRODUCT(QASections_Score[Score],QASections_Score[Importance score])/SUMIF(QASections_Score[Score],\"<>N/A\",QASections_Score[Importance score])","v":0},null,null,null,null,null,null,"A high QA score reflects that both process has been adhered to and quality is present within this process. Each model should aim to score either 90% (if it's business-critical) or 85% if it's not business-critical"]
[null,null,null,null,null,null,null,null,null,null,null,"How is it calculated?"]
[null,null,"Section","Maximum score","Section Score","Score","Importance","Importance score",null,null,null,"The 5 section scores (\"Documentation, \"Structure & Clarity\", \"Verification\", \"Validation\" and \"Data & Assumptions\")  are calculated considering the Priority (\"High\", \"Medium\" or \"Low\") and the Rating (from 1 to 5, with 1 being the best possible score) for each of the tasks considered in the section. The Priority is assigned a priori i.e. is the same for each model) while the Rating is an estimate of the performance of the model against each specific task. A numeric weight corresponds to each level of Priority and Rating. The two weights are then multiplied to get the score of each section's task. The higher the score the poorer the model performance against that task (the best performance scores zero). All the section's tasks are then summed to get the section score. The overall score is then obtained through a weighted sum of the section scores: the 5 QA sections have been assigned an importance rating (\"Essential\", \"Important\" or \"Best Practice) which determines a different weighting. This value is then divided by the worst possible score (i.e. when all the tasks get the worst rating) and this percentage is then inverted (i.e. a score of 43% 'bad' becomes 57% 'good') so that the higher the score the 'better' a QA state for a model is."]
[null,null,"Documentation",{"$f":"SUMIF(Documentation_Score[Rating],\"<>N/A\",Documentation_Score[Priority Score])*Worst_Value","v":490},{"$f":"SUMPRODUCT(--(Documentation_Score[Rating score]<>\"N/A\"),Documentation_Score[Priority Score],Documentation_Score[Rating score])","v":490},{"$f":"IFERROR(1-E10/D10,\"N/A\")","v":0},"Important",{"$f":"INDEX(Importance_Weight[Weight],MATCH(QASections_Score[[#This Row],[Importance]],Importance_Weight[Importance],0))","v":2}]
[null,null,"Structure & Clarity",{"$f":"SUMIF(Structure_Clarity_Score[Rating Score],\"<>N/A\",Structure_Clarity_Score[Priority Score])*Worst_Value","v":340},{"$f":"SUMPRODUCT(--(Structure_Clarity_Score[Rating Score]<>\"N/A\"),Structure_Clarity_Score[Priority Score],Structure_Clarity_Score[Rating Score])","v":340},{"$f":"IFERROR(1-E11/D11,\"N/A\")","v":0},"Best Practice",{"$f":"INDEX(Importance_Weight[Weight],MATCH(QASections_Score[[#This Row],[Importance]],Importance_Weight[Importance],0))","v":1}]
[null,null,"Verification",{"$f":"SUMIF(Verification_Score[Rating Score],\"<>N/A\",Verification_Score[Priority Score])*Worst_Value","v":250},{"$f":"SUMPRODUCT(--(Verification_Score[Rating Score]<>\"N/A\"),Verification_Score[Priority Score],Verification_Score[Rating Score])","v":250},{"$f":"IFERROR(1-E12/D12,\"N/A\")","v":0},"Essential",{"$f":"INDEX(Importance_Weight[Weight],MATCH(QASections_Score[[#This Row],[Importance]],Importance_Weight[Importance],0))","v":3}]
[null,null,"Validation",{"$f":"SUMIF(Validation_Score[Rating Score],\"<>N/A\",Validation_Score[Priority Score])*Worst_Value","v":190},{"$f":"SUMPRODUCT(--(Validation_Score[Rating Score]<>\"N/A\"),Validation_Score[Priority Score],Validation_Score[Rating Score])","v":190},{"$f":"IFERROR(1-E13/D13,\"N/A\")","v":0},"Essential",{"$f":"INDEX(Importance_Weight[Weight],MATCH(QASections_Score[[#This Row],[Importance]],Importance_Weight[Importance],0))","v":3}]
[null,null,"Data & Assumptions",{"$f":"SUMIF(Data_Assumptions_Score[Rating Score],\"<>N/A\",Data_Assumptions_Score[Priority Score])*Worst_Value","v":210},{"$f":"SUMPRODUCT(--(Data_Assumptions_Score[Rating Score]<>\"N/A\"),Data_Assumptions_Score[Priority Score],Data_Assumptions_Score[Rating Score])","v":210},{"$f":"IFERROR(1-E14/D14,\"N/A\")","v":0},"Essential",{"$f":"INDEX(Importance_Weight[Weight],MATCH(QASections_Score[[#This Row],[Importance]],Importance_Weight[Importance],0))","v":3}]
[null,null,null,null,null,null,null,null,null,null,null,"Caveats"]
[null,null,null,null,null,null,null,null,null,null,null,"The model QA score should be a representative metric of a model quality and adherence to accurate QA  processes. However, there are some possible caveats:\n- Different modellers/teams are likely to spend varying levels of time/effort on the template. If the rating assigned to each task is not accurate the overall score will also be imprecise.\n- Although we have tried to provide guidelines to users of this template, there is still a degree of interpretation within\n- The score depends on the weights assigned to each dimension; a change on the weights would change also the overall score"]
[null,null,"Documentation"]
[null,null,"Code","Task","Priority","Priority Score","Rating","Rating score"]
[null,null,"D1","Scope Document","High",{"$f":"IFERROR(INDEX(Priority_Weight[Weight],MATCH(Documentation_Score[[#This Row],[Priority]],Priority_Weight[Priority],0)),\"\")","v":9},{"$f":"IFERROR(INDEX(Documentation[Rating], MATCH(Documentation_Score[[#This Row],[Code]],Documentation[Task Code],0)),\"\")","v":0},{"$f":"IFERROR(IF(Documentation_Score[[#This Row],[Rating]]=0,Worst_Value,INDEX(Rating_Weight[Weight],MATCH(Documentation_Score[[#This Row],[Rating]],Rating_Weight[Rating],0))),\"\")","v":10}]
[null,null,"D2","Spec Document","High",{"$f":"IFERROR(INDEX(Priority_Weight[Weight],MATCH(Documentation_Score[[#This Row],[Priority]],Priority_Weight[Priority],0)),\"\")","v":9},{"$f":"IFERROR(INDEX(Documentation[Rating], MATCH(Documentation_Score[[#This Row],[Code]],Documentation[Task Code],0)),\"\")","v":0},{"$f":"IFERROR(IF(Documentation_Score[[#This Row],[Rating]]=0,Worst_Value,INDEX(Rating_Weight[Weight],MATCH(Documentation_Score[[#This Row],[Rating]],Rating_Weight[Rating],0))),\"\")","v":10}]
[null,null,"D3","User guide","Medium",{"$f":"IFERROR(INDEX(Priority_Weight[Weight],MATCH(Documentation_Score[[#This Row],[Priority]],Priority_Weight[Priority],0)),\"\")","v":3},{"$f":"IFERROR(INDEX(Documentation[Rating], MATCH(Documentation_Score[[#This Row],[Code]],Documentation[Task Code],0)),\"\")","v":0},{"$f":"IFERROR(IF(Documentation_Score[[#This Row],[Rating]]=0,Worst_Value,INDEX(Rating_Weight[Weight],MATCH(Documentation_Score[[#This Row],[Rating]],Rating_Weight[Rating],0))),\"\")","v":10}]
[null,null,"D4","Technical guide","Medium",{"$f":"IFERROR(INDEX(Priority_Weight[Weight],MATCH(Documentation_Score[[#This Row],[Priority]],Priority_Weight[Priority],0)),\"\")","v":3},{"$f":"IFERROR(INDEX(Documentation[Rating], MATCH(Documentation_Score[[#This Row],[Code]],Documentation[Task Code],0)),\"\")","v":0},{"$f":"IFERROR(IF(Documentation_Score[[#This Row],[Rating]]=0,Worst_Value,INDEX(Rating_Weight[Weight],MATCH(Documentation_Score[[#This Row],[Rating]],Rating_Weight[Rating],0))),\"\")","v":10}]
[null,null,"D5","Intranet","Low",{"$f":"IFERROR(INDEX(Priority_Weight[Weight],MATCH(Documentation_Score[[#This Row],[Priority]],Priority_Weight[Priority],0)),\"\")","v":1},{"$f":"IFERROR(INDEX(Documentation[Rating], MATCH(Documentation_Score[[#This Row],[Code]],Documentation[Task Code],0)),\"\")","v":0},{"$f":"IFERROR(IF(Documentation_Score[[#This Row],[Rating]]=0,Worst_Value,INDEX(Rating_Weight[Weight],MATCH(Documentation_Score[[#This Row],[Rating]],Rating_Weight[Rating],0))),\"\")","v":10}]
[null,null,"D6","Version Control","High",{"$f":"IFERROR(INDEX(Priority_Weight[Weight],MATCH(Documentation_Score[[#This Row],[Priority]],Priority_Weight[Priority],0)),\"\")","v":9},{"$f":"IFERROR(INDEX(Documentation[Rating], MATCH(Documentation_Score[[#This Row],[Code]],Documentation[Task Code],0)),\"\")","v":0},{"$f":"IFERROR(IF(Documentation_Score[[#This Row],[Rating]]=0,Worst_Value,INDEX(Rating_Weight[Weight],MATCH(Documentation_Score[[#This Row],[Rating]],Rating_Weight[Rating],0))),\"\")","v":10}]
[null,null,"D7","Governance and responsibilities","Medium",{"$f":"IFERROR(INDEX(Priority_Weight[Weight],MATCH(Documentation_Score[[#This Row],[Priority]],Priority_Weight[Priority],0)),\"\")","v":3},{"$f":"IFERROR(INDEX(Documentation[Rating], MATCH(Documentation_Score[[#This Row],[Code]],Documentation[Task Code],0)),\"\")","v":0},{"$f":"IFERROR(IF(Documentation_Score[[#This Row],[Rating]]=0,Worst_Value,INDEX(Rating_Weight[Weight],MATCH(Documentation_Score[[#This Row],[Rating]],Rating_Weight[Rating],0))),\"\")","v":10}]
[null,null,"D8","Model classification","Medium",{"$f":"IFERROR(INDEX(Priority_Weight[Weight],MATCH(Documentation_Score[[#This Row],[Priority]],Priority_Weight[Priority],0)),\"\")","v":3},{"$f":"IFERROR(INDEX(Documentation[Rating], MATCH(Documentation_Score[[#This Row],[Code]],Documentation[Task Code],0)),\"\")","v":0},{"$f":"IFERROR(IF(Documentation_Score[[#This Row],[Rating]]=0,Worst_Value,INDEX(Rating_Weight[Weight],MATCH(Documentation_Score[[#This Row],[Rating]],Rating_Weight[Rating],0))),\"\")","v":10}]
[null,null,"D9","QA planning and resourcing","High",{"$f":"IFERROR(INDEX(Priority_Weight[Weight],MATCH(Documentation_Score[[#This Row],[Priority]],Priority_Weight[Priority],0)),\"\")","v":9},{"$f":"IFERROR(INDEX(Documentation[Rating], MATCH(Documentation_Score[[#This Row],[Code]],Documentation[Task Code],0)),\"\")","v":0},{"$f":"IFERROR(IF(Documentation_Score[[#This Row],[Rating]]=0,Worst_Value,INDEX(Rating_Weight[Weight],MATCH(Documentation_Score[[#This Row],[Rating]],Rating_Weight[Rating],0))),\"\")","v":10}]
[null,null,"D10",null,null,{"$f":"IFERROR(INDEX(Priority_Weight[Weight],MATCH(Documentation_Score[[#This Row],[Priority]],Priority_Weight[Priority],0)),\"\")","v":""},{"$f":"IFERROR(INDEX(Documentation[Rating], MATCH(Documentation_Score[[#This Row],[Code]],Documentation[Task Code],0)),\"\")","v":""},{"$f":"IFERROR(IF(Documentation_Score[[#This Row],[Rating]]=0,Worst_Value,INDEX(Rating_Weight[Weight],MATCH(Documentation_Score[[#This Row],[Rating]],Rating_Weight[Rating],0))),\"\")","v":""}]
[]
[null,null,"Structure & Best Practice"]
[null,null,"Code","Task","Priority","Priority Score","Rating","Rating Score"]
[null,null,"S1","Model structure","Medium",{"$f":"IFERROR(INDEX(Priority_Weight[Weight],MATCH(Structure_Clarity_Score[[#This Row],[Priority]],Priority_Weight[Priority],0)),\"\")","v":3},{"$f":"IFERROR(INDEX(Structure_Clarity[Rating], MATCH(Structure_Clarity_Score[[#This Row],[Code]],Structure_Clarity[Task Code],0)),\"\")","v":0},{"$f":"IFERROR(IF(Structure_Clarity_Score[[#This Row],[Rating]]=0,Worst_Value,INDEX(Rating_Weight[Weight],MATCH(Structure_Clarity_Score[[#This Row],[Rating]],Rating_Weight[Rating],0))),\"\")","v":10}]
[null,null,"S2","Worksheets structure ","Medium",{"$f":"IFERROR(INDEX(Priority_Weight[Weight],MATCH(Structure_Clarity_Score[[#This Row],[Priority]],Priority_Weight[Priority],0)),\"\")","v":3},{"$f":"IFERROR(INDEX(Structure_Clarity[Rating], MATCH(Structure_Clarity_Score[[#This Row],[Code]],Structure_Clarity[Task Code],0)),\"\")","v":0},{"$f":"IFERROR(IF(Structure_Clarity_Score[[#This Row],[Rating]]=0,Worst_Value,INDEX(Rating_Weight[Weight],MATCH(Structure_Clarity_Score[[#This Row],[Rating]],Rating_Weight[Rating],0))),\"\")","v":10}]
[null,null,"S3","Labels, Units & Conversions","High",{"$f":"IFERROR(INDEX(Priority_Weight[Weight],MATCH(Structure_Clarity_Score[[#This Row],[Priority]],Priority_Weight[Priority],0)),\"\")","v":9},{"$f":"IFERROR(INDEX(Structure_Clarity[Rating], MATCH(Structure_Clarity_Score[[#This Row],[Code]],Structure_Clarity[Task Code],0)),\"\")","v":0},{"$f":"IFERROR(IF(Structure_Clarity_Score[[#This Row],[Rating]]=0,Worst_Value,INDEX(Rating_Weight[Weight],MATCH(Structure_Clarity_Score[[#This Row],[Rating]],Rating_Weight[Rating],0))),\"\")","v":10}]
[null,null,"S4","Formatting","Medium",{"$f":"IFERROR(INDEX(Priority_Weight[Weight],MATCH(Structure_Clarity_Score[[#This Row],[Priority]],Priority_Weight[Priority],0)),\"\")","v":3},{"$f":"IFERROR(INDEX(Structure_Clarity[Rating], MATCH(Structure_Clarity_Score[[#This Row],[Code]],Structure_Clarity[Task Code],0)),\"\")","v":0},{"$f":"IFERROR(IF(Structure_Clarity_Score[[#This Row],[Rating]]=0,Worst_Value,INDEX(Rating_Weight[Weight],MATCH(Structure_Clarity_Score[[#This Row],[Rating]],Rating_Weight[Rating],0))),\"\")","v":10}]
[null,null,"S5","Workbook Comments","Medium",{"$f":"IFERROR(INDEX(Priority_Weight[Weight],MATCH(Structure_Clarity_Score[[#This Row],[Priority]],Priority_Weight[Priority],0)),\"\")","v":3},{"$f":"IFERROR(INDEX(Structure_Clarity[Rating], MATCH(Structure_Clarity_Score[[#This Row],[Code]],Structure_Clarity[Task Code],0)),\"\")","v":0},{"$f":"IFERROR(IF(Structure_Clarity_Score[[#This Row],[Rating]]=0,Worst_Value,INDEX(Rating_Weight[Weight],MATCH(Structure_Clarity_Score[[#This Row],[Rating]],Rating_Weight[Rating],0))),\"\")","v":10}]
[null,null,"S6","Code Comments","High",{"$f":"IFERROR(INDEX(Priority_Weight[Weight],MATCH(Structure_Clarity_Score[[#This Row],[Priority]],Priority_Weight[Priority],0)),\"\")","v":9},{"$f":"IFERROR(INDEX(Structure_Clarity[Rating], MATCH(Structure_Clarity_Score[[#This Row],[Code]],Structure_Clarity[Task Code],0)),\"\")","v":0},{"$f":"IFERROR(IF(Structure_Clarity_Score[[#This Row],[Rating]]=0,Worst_Value,INDEX(Rating_Weight[Weight],MATCH(Structure_Clarity_Score[[#This Row],[Rating]],Rating_Weight[Rating],0))),\"\")","v":10}]
[null,null,"S7","Formula clarity & robustness","Medium",{"$f":"IFERROR(INDEX(Priority_Weight[Weight],MATCH(Structure_Clarity_Score[[#This Row],[Priority]],Priority_Weight[Priority],0)),\"\")","v":3},{"$f":"IFERROR(INDEX(Structure_Clarity[Rating], MATCH(Structure_Clarity_Score[[#This Row],[Code]],Structure_Clarity[Task Code],0)),\"\")","v":0},{"$f":"IFERROR(IF(Structure_Clarity_Score[[#This Row],[Rating]]=0,Worst_Value,INDEX(Rating_Weight[Weight],MATCH(Structure_Clarity_Score[[#This Row],[Rating]],Rating_Weight[Rating],0))),\"\")","v":10}]
[null,null,"S8","Named ranges management","Low",{"$f":"IFERROR(INDEX(Priority_Weight[Weight],MATCH(Structure_Clarity_Score[[#This Row],[Priority]],Priority_Weight[Priority],0)),\"\")","v":1},{"$f":"IFERROR(INDEX(Structure_Clarity[Rating], MATCH(Structure_Clarity_Score[[#This Row],[Code]],Structure_Clarity[Task Code],0)),\"\")","v":0},{"$f":"IFERROR(IF(Structure_Clarity_Score[[#This Row],[Rating]]=0,Worst_Value,INDEX(Rating_Weight[Weight],MATCH(Structure_Clarity_Score[[#This Row],[Rating]],Rating_Weight[Rating],0))),\"\")","v":10}]
[null,null,"S9",null,null,{"$f":"IFERROR(INDEX(Priority_Weight[Weight],MATCH(Structure_Clarity_Score[[#This Row],[Priority]],Priority_Weight[Priority],0)),\"\")","v":""},{"$f":"IFERROR(INDEX(Structure_Clarity[Rating], MATCH(Structure_Clarity_Score[[#This Row],[Code]],Structure_Clarity[Task Code],0)),\"\")","v":""},{"$f":"IFERROR(IF(Structure_Clarity_Score[[#This Row],[Rating]]=0,Worst_Value,INDEX(Rating_Weight[Weight],MATCH(Structure_Clarity_Score[[#This Row],[Rating]],Rating_Weight[Rating],0))),\"\")","v":""}]
[null,null,"S10",null,null,null,{"$f":"IFERROR(INDEX(Structure_Clarity[Rating], MATCH(Structure_Clarity_Score[[#This Row],[Code]],Structure_Clarity[Task Code],0)),\"\")","v":""},{"$f":"IFERROR(IF(Structure_Clarity_Score[[#This Row],[Rating]]=0,Worst_Value,INDEX(Rating_Weight[Weight],MATCH(Structure_Clarity_Score[[#This Row],[Rating]],Rating_Weight[Rating],0))),\"\")","v":""}]
[]
[null,null,"Verification"]
[null,null,"Code","Task","Priority","Priority Score","Rating","Rating Score"]
[null,null,"Ve1","Formula correctness","High",{"$f":"IFERROR(INDEX(Priority_Weight[Weight],MATCH(Verification_Score[[#This Row],[Priority]],Priority_Weight[Priority],0)),\"\")","v":9},{"$f":"IFERROR(INDEX(Verification[Rating], MATCH(Verification_Score[[#This Row],[Code]],Verification[Task Code],0)),\"\")","v":0},{"$f":"IFERROR(IF(Verification_Score[[#This Row],[Rating]]=0,Worst_Value,INDEX(Rating_Weight[Weight],MATCH(Verification_Score[[#This Row],[Rating]],Rating_Weight[Rating],0))),\"\")","v":10}]
[null,null,"Ve2","Usability Testing","Medium",{"$f":"IFERROR(INDEX(Priority_Weight[Weight],MATCH(Verification_Score[[#This Row],[Priority]],Priority_Weight[Priority],0)),\"\")","v":3},{"$f":"IFERROR(INDEX(Verification[Rating], MATCH(Verification_Score[[#This Row],[Code]],Verification[Task Code],0)),\"\")","v":0},{"$f":"IFERROR(IF(Verification_Score[[#This Row],[Rating]]=0,Worst_Value,INDEX(Rating_Weight[Weight],MATCH(Verification_Score[[#This Row],[Rating]],Rating_Weight[Rating],0))),\"\")","v":10}]
[null,null,"Ve3","Code correctness","High",{"$f":"IFERROR(INDEX(Priority_Weight[Weight],MATCH(Verification_Score[[#This Row],[Priority]],Priority_Weight[Priority],0)),\"\")","v":9},{"$f":"IFERROR(INDEX(Verification[Rating], MATCH(Verification_Score[[#This Row],[Code]],Verification[Task Code],0)),\"\")","v":0},{"$f":"IFERROR(IF(Verification_Score[[#This Row],[Rating]]=0,Worst_Value,INDEX(Rating_Weight[Weight],MATCH(Verification_Score[[#This Row],[Rating]],Rating_Weight[Rating],0))),\"\")","v":10}]
[null,null,"Ve4","External links","Medium",{"$f":"IFERROR(INDEX(Priority_Weight[Weight],MATCH(Verification_Score[[#This Row],[Priority]],Priority_Weight[Priority],0)),\"\")","v":3},{"$f":"IFERROR(INDEX(Verification[Rating], MATCH(Verification_Score[[#This Row],[Code]],Verification[Task Code],0)),\"\")","v":0},{"$f":"IFERROR(IF(Verification_Score[[#This Row],[Rating]]=0,Worst_Value,INDEX(Rating_Weight[Weight],MATCH(Verification_Score[[#This Row],[Rating]],Rating_Weight[Rating],0))),\"\")","v":10}]
[null,null,"Ve5","Autochecks & Regression Testing","Low",{"$f":"IFERROR(INDEX(Priority_Weight[Weight],MATCH(Verification_Score[[#This Row],[Priority]],Priority_Weight[Priority],0)),\"\")","v":1},{"$f":"IFERROR(INDEX(Verification[Rating], MATCH(Verification_Score[[#This Row],[Code]],Verification[Task Code],0)),\"\")","v":0},{"$f":"IFERROR(IF(Verification_Score[[#This Row],[Rating]]=0,Worst_Value,INDEX(Rating_Weight[Weight],MATCH(Verification_Score[[#This Row],[Rating]],Rating_Weight[Rating],0))),\"\")","v":10}]
[null,null,"Ve6"]
[null,null,"Ve7"]
[null,null,"Ve8"]
[null,null,"Ve9",null,null,{"$f":"IFERROR(INDEX(Priority_Weight[Weight],MATCH(Verification_Score[[#This Row],[Priority]],Priority_Weight[Priority],0)),\"\")","v":""},{"$f":"IFERROR(INDEX(Verification[Rating], MATCH(Verification_Score[[#This Row],[Code]],Verification[Task Code],0)),\"\")","v":""},{"$f":"IFERROR(IF(Verification_Score[[#This Row],[Rating]]=0,Worst_Value,INDEX(Rating_Weight[Weight],MATCH(Verification_Score[[#This Row],[Rating]],Rating_Weight[Rating],0))),\"\")","v":""}]
[null,null,"Ve10",null,null,{"$f":"IFERROR(INDEX(Priority_Weight[Weight],MATCH(Verification_Score[[#This Row],[Priority]],Priority_Weight[Priority],0)),\"\")","v":""},{"$f":"IFERROR(INDEX(Verification[Rating], MATCH(Verification_Score[[#This Row],[Code]],Verification[Task Code],0)),\"\")","v":""},{"$f":"IFERROR(IF(Verification_Score[[#This Row],[Rating]]=0,Worst_Value,INDEX(Rating_Weight[Weight],MATCH(Verification_Score[[#This Row],[Rating]],Rating_Weight[Rating],0))),\"\")","v":""}]
[]
[null,null,"Validation"]
[null,null,"Code","Task","Priority","Priority Score","Rating","Rating Score"]
[null,null,"Va1","Methodology correctness","High",{"$f":"IFERROR(INDEX(Priority_Weight[Weight],MATCH(Validation_Score[[#This Row],[Priority]],Priority_Weight[Priority],0)),\"\")","v":9},{"$f":"IFERROR(INDEX(Validation[Rating], MATCH(Validation_Score[[#This Row],[Code]],Validation[Task Code],0)),\"\")","v":0},{"$f":"IFERROR(IF(Validation_Score[[#This Row],[Rating]]=0,Worst_Value,INDEX(Rating_Weight[Weight],MATCH(Validation_Score[[#This Row],[Rating]],Rating_Weight[Rating],0))),\"\")","v":10}]
[null,null,"Va2","Comparison with historical data / backcasting / alternative models","Medium",{"$f":"IFERROR(INDEX(Priority_Weight[Weight],MATCH(Validation_Score[[#This Row],[Priority]],Priority_Weight[Priority],0)),\"\")","v":3},{"$f":"IFERROR(INDEX(Validation[Rating], MATCH(Validation_Score[[#This Row],[Code]],Validation[Task Code],0)),\"\")","v":0},{"$f":"IFERROR(IF(Validation_Score[[#This Row],[Rating]]=0,Worst_Value,INDEX(Rating_Weight[Weight],MATCH(Validation_Score[[#This Row],[Rating]],Rating_Weight[Rating],0))),\"\")","v":10}]
[null,null,"Va3","Uncertainty testing","Medium",{"$f":"IFERROR(INDEX(Priority_Weight[Weight],MATCH(Validation_Score[[#This Row],[Priority]],Priority_Weight[Priority],0)),\"\")","v":3},{"$f":"IFERROR(INDEX(Validation[Rating], MATCH(Validation_Score[[#This Row],[Code]],Validation[Task Code],0)),\"\")","v":0},{"$f":"IFERROR(IF(Validation_Score[[#This Row],[Rating]]=0,Worst_Value,INDEX(Rating_Weight[Weight],MATCH(Validation_Score[[#This Row],[Rating]],Rating_Weight[Rating],0))),\"\")","v":10}]
[null,null,"Va4","Extreme values testing / model breaking","Medium",{"$f":"IFERROR(INDEX(Priority_Weight[Weight],MATCH(Validation_Score[[#This Row],[Priority]],Priority_Weight[Priority],0)),\"\")","v":3},{"$f":"IFERROR(INDEX(Validation[Rating], MATCH(Validation_Score[[#This Row],[Code]],Validation[Task Code],0)),\"\")","v":0},{"$f":"IFERROR(IF(Validation_Score[[#This Row],[Rating]]=0,Worst_Value,INDEX(Rating_Weight[Weight],MATCH(Validation_Score[[#This Row],[Rating]],Rating_Weight[Rating],0))),\"\")","v":10}]
[null,null,"Va5","Re-performance testing","Low",{"$f":"IFERROR(INDEX(Priority_Weight[Weight],MATCH(Validation_Score[[#This Row],[Priority]],Priority_Weight[Priority],0)),\"\")","v":1},{"$f":"IFERROR(INDEX(Validation[Rating], MATCH(Validation_Score[[#This Row],[Code]],Validation[Task Code],0)),\"\")","v":0},{"$f":"IFERROR(IF(Validation_Score[[#This Row],[Rating]]=0,Worst_Value,INDEX(Rating_Weight[Weight],MATCH(Validation_Score[[#This Row],[Rating]],Rating_Weight[Rating],0))),\"\")","v":10}]
[null,null,"Va6",null,null,null,{"$f":"IFERROR(INDEX(Validation[Rating], MATCH(Validation_Score[[#This Row],[Code]],Validation[Task Code],0)),\"\")","v":""},{"$f":"IFERROR(IF(Validation_Score[[#This Row],[Rating]]=0,Worst_Value,INDEX(Rating_Weight[Weight],MATCH(Validation_Score[[#This Row],[Rating]],Rating_Weight[Rating],0))),\"\")","v":""}]
[null,null,"Va7",null,null,{"$f":"IFERROR(INDEX(Priority_Weight[Weight],MATCH(Validation_Score[[#This Row],[Priority]],Priority_Weight[Priority],0)),\"\")","v":""},{"$f":"IFERROR(INDEX(Validation[Rating], MATCH(Validation_Score[[#This Row],[Code]],Validation[Task Code],0)),\"\")","v":""},{"$f":"IFERROR(IF(Validation_Score[[#This Row],[Rating]]=0,Worst_Value,INDEX(Rating_Weight[Weight],MATCH(Validation_Score[[#This Row],[Rating]],Rating_Weight[Rating],0))),\"\")","v":""}]
[null,null,"Va8",null,null,{"$f":"IFERROR(INDEX(Priority_Weight[Weight],MATCH(Validation_Score[[#This Row],[Priority]],Priority_Weight[Priority],0)),\"\")","v":""},{"$f":"IFERROR(INDEX(Validation[Rating], MATCH(Validation_Score[[#This Row],[Code]],Validation[Task Code],0)),\"\")","v":""},{"$f":"IFERROR(IF(Validation_Score[[#This Row],[Rating]]=0,Worst_Value,INDEX(Rating_Weight[Weight],MATCH(Validation_Score[[#This Row],[Rating]],Rating_Weight[Rating],0))),\"\")","v":""}]
[null,null,"Va9",null,null,{"$f":"IFERROR(INDEX(Priority_Weight[Weight],MATCH(Validation_Score[[#This Row],[Priority]],Priority_Weight[Priority],0)),\"\")","v":""},{"$f":"IFERROR(INDEX(Validation[Rating], MATCH(Validation_Score[[#This Row],[Code]],Validation[Task Code],0)),\"\")","v":""},{"$f":"IFERROR(IF(Validation_Score[[#This Row],[Rating]]=0,Worst_Value,INDEX(Rating_Weight[Weight],MATCH(Validation_Score[[#This Row],[Rating]],Rating_Weight[Rating],0))),\"\")","v":""}]
[null,null,"Va10",null,null,{"$f":"IFERROR(INDEX(Priority_Weight[Weight],MATCH(Validation_Score[[#This Row],[Priority]],Priority_Weight[Priority],0)),\"\")","v":""},{"$f":"IFERROR(INDEX(Validation[Rating], MATCH(Validation_Score[[#This Row],[Code]],Validation[Task Code],0)),\"\")","v":""},{"$f":"IFERROR(IF(Validation_Score[[#This Row],[Rating]]=0,Worst_Value,INDEX(Rating_Weight[Weight],MATCH(Validation_Score[[#This Row],[Rating]],Rating_Weight[Rating],0))),\"\")","v":""}]
[]
[null,null,"Data & Assumptions"]
[null,null,"Code","Task","Priority","Priority Score","Rating","Rating Score"]
[null,null,"DA1","Data","High",{"$f":"IFERROR(INDEX(Priority_Weight[Weight],MATCH(Data_Assumptions_Score[[#This Row],[Priority]],Priority_Weight[Priority],0)),\"\")","v":9},{"$f":"IFERROR(INDEX(Data_Assumptions[Rating], MATCH(Data_Assumptions_Score[[#This Row],[Code]],Data_Assumptions[Task Code],0)),\"\")","v":0},{"$f":"IFERROR(IF(Data_Assumptions_Score[[#This Row],[Rating]]=0,Worst_Value,INDEX(Rating_Weight[Weight],MATCH(Data_Assumptions_Score[[#This Row],[Rating]],Rating_Weight[Rating],0))),\"\")","v":10}]
[null,null,"DA2","Data transformation","Medium",{"$f":"IFERROR(INDEX(Priority_Weight[Weight],MATCH(Data_Assumptions_Score[[#This Row],[Priority]],Priority_Weight[Priority],0)),\"\")","v":3},{"$f":"IFERROR(INDEX(Data_Assumptions[Rating], MATCH(Data_Assumptions_Score[[#This Row],[Code]],Data_Assumptions[Task Code],0)),\"\")","v":0},{"$f":"IFERROR(IF(Data_Assumptions_Score[[#This Row],[Rating]]=0,Worst_Value,INDEX(Rating_Weight[Weight],MATCH(Data_Assumptions_Score[[#This Row],[Rating]],Rating_Weight[Rating],0))),\"\")","v":10}]
[null,null,"DA3","Assumptions","High",{"$f":"IFERROR(INDEX(Priority_Weight[Weight],MATCH(Data_Assumptions_Score[[#This Row],[Priority]],Priority_Weight[Priority],0)),\"\")","v":9},{"$f":"IFERROR(INDEX(Data_Assumptions[Rating], MATCH(Data_Assumptions_Score[[#This Row],[Code]],Data_Assumptions[Task Code],0)),\"\")","v":0},{"$f":"IFERROR(IF(Data_Assumptions_Score[[#This Row],[Rating]]=0,Worst_Value,INDEX(Rating_Weight[Weight],MATCH(Data_Assumptions_Score[[#This Row],[Rating]],Rating_Weight[Rating],0))),\"\")","v":10}]
[null,null,"DA4",null,null,{"$f":"IFERROR(INDEX(Priority_Weight[Weight],MATCH(Data_Assumptions_Score[[#This Row],[Priority]],Priority_Weight[Priority],0)),\"\")","v":""},{"$f":"IFERROR(INDEX(Data_Assumptions[Rating], MATCH(Data_Assumptions_Score[[#This Row],[Code]],Data_Assumptions[Task Code],0)),\"\")","v":""},{"$f":"IFERROR(IF(Data_Assumptions_Score[[#This Row],[Rating]]=0,Worst_Value,INDEX(Rating_Weight[Weight],MATCH(Data_Assumptions_Score[[#This Row],[Rating]],Rating_Weight[Rating],0))),\"\")","v":""}]
[null,null,"DA5",null,null,{"$f":"IFERROR(INDEX(Priority_Weight[Weight],MATCH(Data_Assumptions_Score[[#This Row],[Priority]],Priority_Weight[Priority],0)),\"\")","v":""},{"$f":"IFERROR(INDEX(Data_Assumptions[Rating], MATCH(Data_Assumptions_Score[[#This Row],[Code]],Data_Assumptions[Task Code],0)),\"\")","v":""},{"$f":"IFERROR(IF(Data_Assumptions_Score[[#This Row],[Rating]]=0,Worst_Value,INDEX(Rating_Weight[Weight],MATCH(Data_Assumptions_Score[[#This Row],[Rating]],Rating_Weight[Rating],0))),\"\")","v":""}]
[null,null,"DA6",null,null,{"$f":"IFERROR(INDEX(Priority_Weight[Weight],MATCH(Data_Assumptions_Score[[#This Row],[Priority]],Priority_Weight[Priority],0)),\"\")","v":""},{"$f":"IFERROR(INDEX(Data_Assumptions[Rating], MATCH(Data_Assumptions_Score[[#This Row],[Code]],Data_Assumptions[Task Code],0)),\"\")","v":""},{"$f":"IFERROR(IF(Data_Assumptions_Score[[#This Row],[Rating]]=0,Worst_Value,INDEX(Rating_Weight[Weight],MATCH(Data_Assumptions_Score[[#This Row],[Rating]],Rating_Weight[Rating],0))),\"\")","v":""}]
[null,null,"DA7",null,null,{"$f":"IFERROR(INDEX(Priority_Weight[Weight],MATCH(Data_Assumptions_Score[[#This Row],[Priority]],Priority_Weight[Priority],0)),\"\")","v":""},{"$f":"IFERROR(INDEX(Data_Assumptions[Rating], MATCH(Data_Assumptions_Score[[#This Row],[Code]],Data_Assumptions[Task Code],0)),\"\")","v":""},{"$f":"IFERROR(IF(Data_Assumptions_Score[[#This Row],[Rating]]=0,Worst_Value,INDEX(Rating_Weight[Weight],MATCH(Data_Assumptions_Score[[#This Row],[Rating]],Rating_Weight[Rating],0))),\"\")","v":""}]
[null,null,"DA8",null,null,{"$f":"IFERROR(INDEX(Priority_Weight[Weight],MATCH(Data_Assumptions_Score[[#This Row],[Priority]],Priority_Weight[Priority],0)),\"\")","v":""},{"$f":"IFERROR(INDEX(Data_Assumptions[Rating], MATCH(Data_Assumptions_Score[[#This Row],[Code]],Data_Assumptions[Task Code],0)),\"\")","v":""},{"$f":"IFERROR(IF(Data_Assumptions_Score[[#This Row],[Rating]]=0,Worst_Value,INDEX(Rating_Weight[Weight],MATCH(Data_Assumptions_Score[[#This Row],[Rating]],Rating_Weight[Rating],0))),\"\")","v":""}]
[null,null,"DA9",null,null,{"$f":"IFERROR(INDEX(Priority_Weight[Weight],MATCH(Data_Assumptions_Score[[#This Row],[Priority]],Priority_Weight[Priority],0)),\"\")","v":""},{"$f":"IFERROR(INDEX(Data_Assumptions[Rating], MATCH(Data_Assumptions_Score[[#This Row],[Code]],Data_Assumptions[Task Code],0)),\"\")","v":""},{"$f":"IFERROR(IF(Data_Assumptions_Score[[#This Row],[Rating]]=0,Worst_Value,INDEX(Rating_Weight[Weight],MATCH(Data_Assumptions_Score[[#This Row],[Rating]],Rating_Weight[Rating],0))),\"\")","v":""}]
[null,null,"DA10",null,null,{"$f":"IFERROR(INDEX(Priority_Weight[Weight],MATCH(Data_Assumptions_Score[[#This Row],[Priority]],Priority_Weight[Priority],0)),\"\")","v":""},{"$f":"IFERROR(INDEX(Data_Assumptions[Rating], MATCH(Data_Assumptions_Score[[#This Row],[Code]],Data_Assumptions[Task Code],0)),\"\")","v":""},{"$f":"IFERROR(IF(Data_Assumptions_Score[[#This Row],[Rating]]=0,Worst_Value,INDEX(Rating_Weight[Weight],MATCH(Data_Assumptions_Score[[#This Row],[Rating]],Rating_Weight[Rating],0))),\"\")","v":""}]

{"sheet":"QA plan","index":9,"n_rows":42,"n_cols":13,"rows_kept":42,"cut":false,"hidden":false,"merged":[[4,1,4,12]]}
[]
[{"$f":"IF(SecurityClassification=\"OFFICIAL\",\"\",SecurityClassification & \" - \")  & DocumentName & \" QA log - version \" & VersionNumber","v":" -  QA log - version "}]
[]
[]
[null,"This sheet must be used to list QA activities that will be carried out on the model in order to reach and mantain a QA score of 90% or more. \nThe QA activities are taken from the orange sheets of this file: you can use filters on the \"Score before this QA task was undertaken\" to hide the tasks which have already been performed (i.e. those rated as Excellent). For the remaining tasks, you should fill all the columns in the table below. "]
[null,"The end of the next financial year is highlighted in green. This table excludes rating improvements from overdue tasks (i.e. tasks where the completion date has passed, but the target score has not yet been reached)"]
[null,null,{"$f":"TODAY()","v":"2018-06-18"},{"$f":"MIN(DATE(YEAR(C7+100),MONTH(C7+100),1),DATE(IF(MONTH(C7)<4,YEAR(C7),YEAR(C7)+1),4,1))","v":"2018-09-01"},{"$f":"MIN(DATE(YEAR(D7+100),MONTH(D7+100),1),DATE(IF(MONTH(D7)<4,YEAR(D7),YEAR(D7)+1),4,1))","v":"2018-12-01"},{"$f":"MIN(DATE(YEAR(E7+100),MONTH(E7+100),1),DATE(IF(MONTH(E7)<4,YEAR(E7),YEAR(E7)+1),4,1))","v":"2019-03-01"},{"$f":"MIN(DATE(YEAR(F7+100),MONTH(F7+100),1),DATE(IF(MONTH(F7)<4,YEAR(F7),YEAR(F7)+1),4,1))","v":"2019-04-01"},{"$f":"MIN(DATE(YEAR(G7+100),MONTH(G7+100),1),DATE(IF(MONTH(G7)<4,YEAR(G7),YEAR(G7)+1),4,1))","v":"2019-07-01"}]
[null,"Autocalculated QA score",{"$f":"Review_Overall_Score","v":0},{"$f":"OriginalScore+SUMIFS(Table1[Score change],Table1[Completion date],\"<\" & D7,Table1[Completion date],\">=\"&TODAY())","v":0},{"$f":"OriginalScore+SUMIFS(Table1[Score change],Table1[Completion date],\"<\" & E7,Table1[Completion date],\">=\"&TODAY())","v":0},{"$f":"OriginalScore+SUMIFS(Table1[Score change],Table1[Completion date],\"<\" & F7,Table1[Completion date],\">=\"&TODAY())","v":0},{"$f":"OriginalScore+SUMIFS(Table1[Score change],Table1[Completion date],\"<\" & G7,Table1[Completion date],\">=\"&TODAY())","v":0},{"$f":"OriginalScore+SUMIFS(Table1[Score change],Table1[Completion date],\"<\" & H7,Table1[Completion date],\">=\"&TODAY())","v":0}]
[null,"Please add any comments"]
[]
[null,"QA section","Linked to the QA log","As written in the QA log","Describe the QA task to perform","An assessment of the importance of the task for the model","Person who will perform the task","Insert estimated time required to complete the task","Estimated date when the QA activity has been completed. ROWS ARE HIGHLIGHTED IN RED IF THE DATE IS IN THE PAST","Date when MIT checked the task was completed","Taken from the QA log, change the rating there when the task is complete","Look at the ratings guidelines in the orange tabs\nROWS ARE HIGHLIGHTED IN GREEN IF THE SCORE IS REACHED","Change (percentage points) in the overall score when the task is complete"]
[null,"Section","QA task ","Outstanding work","Task description","Priority","Name","Person hours required","Completion date","Checked by MIT","Score before this QA task was undertaken","Expected score after this QA is done","Score change"]
[null,"Documentation","Scope Document",{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Outstanding Work]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":"Must\n\nShould\n\nCould"},null,null,null,null,null,null,{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Rating]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":""},null,{"$f":"IFERROR(INDEX(QASections_Score[Importance score],MATCH(Table1[[#This Row],[Section]],QASections_Score[Section],0))/SUM(QASections_Score[Importance score])*INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Priority Score]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0))/SUMIF(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Rating]\"),\"<>N/A\",INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Priority Score]\"))*(INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Score before this QA task was undertaken]],Rating_Weight[Rating],0))-INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Expected score after this QA is done]],Rating_Weight[Rating],0)))/10,0)","v":0}]
[null,"Documentation","Spec Document",{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Outstanding Work]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":"Must\n\nShould\n\nCould"},null,null,null,null,null,null,{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Rating]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":""},null,{"$f":"IFERROR(INDEX(QASections_Score[Importance score],MATCH(Table1[[#This Row],[Section]],QASections_Score[Section],0))/SUM(QASections_Score[Importance score])*INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Priority Score]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0))/SUMIF(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Rating]\"),\"<>N/A\",INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Priority Score]\"))*(INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Score before this QA task was undertaken]],Rating_Weight[Rating],0))-INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Expected score after this QA is done]],Rating_Weight[Rating],0)))/10,0)","v":0}]
[null,"Documentation","User guide",{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Outstanding Work]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":"Must\n\nShould\n\nCould"},null,null,null,null,null,null,{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Rating]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":""},null,{"$f":"IFERROR(INDEX(QASections_Score[Importance score],MATCH(Table1[[#This Row],[Section]],QASections_Score[Section],0))/SUM(QASections_Score[Importance score])*INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Priority Score]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0))/SUMIF(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Rating]\"),\"<>N/A\",INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Priority Score]\"))*(INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Score before this QA task was undertaken]],Rating_Weight[Rating],0))-INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Expected score after this QA is done]],Rating_Weight[Rating],0)))/10,0)","v":0}]
[null,"Documentation","Technical guide",{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Outstanding Work]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":"Must\n\nShould\n\nCould"},null,null,null,null,null,null,{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Rating]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":""},null,{"$f":"IFERROR(INDEX(QASections_Score[Importance score],MATCH(Table1[[#This Row],[Section]],QASections_Score[Section],0))/SUM(QASections_Score[Importance score])*INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Priority Score]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0))/SUMIF(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Rating]\"),\"<>N/A\",INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Priority Score]\"))*(INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Score before this QA task was undertaken]],Rating_Weight[Rating],0))-INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Expected score after this QA is done]],Rating_Weight[Rating],0)))/10,0)","v":0}]
[null,"Documentation","Intranet",{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Outstanding Work]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":"Must\n\nShould\n\nCould"},null,null,null,null,null,null,{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Rating]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":""},null,{"$f":"IFERROR(INDEX(QASections_Score[Importance score],MATCH(Table1[[#This Row],[Section]],QASections_Score[Section],0))/SUM(QASections_Score[Importance score])*INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Priority Score]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0))/SUMIF(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Rating]\"),\"<>N/A\",INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Priority Score]\"))*(INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Score before this QA task was undertaken]],Rating_Weight[Rating],0))-INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Expected score after this QA is done]],Rating_Weight[Rating],0)))/10,0)","v":0}]
[null,"Documentation","Version Control",{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Outstanding Work]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":"Must\n\nShould\n\nCould"},null,null,null,null,null,null,{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Rating]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":""},null,{"$f":"IFERROR(INDEX(QASections_Score[Importance score],MATCH(Table1[[#This Row],[Section]],QASections_Score[Section],0))/SUM(QASections_Score[Importance score])*INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Priority Score]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0))/SUMIF(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Rating]\"),\"<>N/A\",INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Priority Score]\"))*(INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Score before this QA task was undertaken]],Rating_Weight[Rating],0))-INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Expected score after this QA is done]],Rating_Weight[Rating],0)))/10,0)","v":0}]
[null,"Documentation","Governance and responsibilities",{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Outstanding Work]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":"Must\n\nShould\n\nCould"},null,null,null,null,null,null,{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Rating]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":""},null,{"$f":"IFERROR(INDEX(QASections_Score[Importance score],MATCH(Table1[[#This Row],[Section]],QASections_Score[Section],0))/SUM(QASections_Score[Importance score])*INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Priority Score]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0))/SUMIF(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Rating]\"),\"<>N/A\",INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Priority Score]\"))*(INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Score before this QA task was undertaken]],Rating_Weight[Rating],0))-INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Expected score after this QA is done]],Rating_Weight[Rating],0)))/10,0)","v":0}]
[null,"Documentation","Model classification",{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Outstanding Work]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":"Must\n\nShould\n\nCould"},null,null,null,null,null,null,{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Rating]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":""},null,{"$f":"IFERROR(INDEX(QASections_Score[Importance score],MATCH(Table1[[#This Row],[Section]],QASections_Score[Section],0))/SUM(QASections_Score[Importance score])*INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Priority Score]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0))/SUMIF(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Rating]\"),\"<>N/A\",INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Priority Score]\"))*(INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Score before this QA task was undertaken]],Rating_Weight[Rating],0))-INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Expected score after this QA is done]],Rating_Weight[Rating],0)))/10,0)","v":0}]
[null,"Documentation","QA planning and resourcing",{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Outstanding Work]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":"Must\n\nShould\n\nCould"},null,null,null,null,null,null,{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Rating]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":""},null,{"$f":"IFERROR(INDEX(QASections_Score[Importance score],MATCH(Table1[[#This Row],[Section]],QASections_Score[Section],0))/SUM(QASections_Score[Importance score])*INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Priority Score]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0))/SUMIF(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Rating]\"),\"<>N/A\",INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Priority Score]\"))*(INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Score before this QA task was undertaken]],Rating_Weight[Rating],0))-INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Expected score after this QA is done]],Rating_Weight[Rating],0)))/10,0)","v":0}]
[null,"Structure & Clarity","Model structure",{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Outstanding Work]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":"Must\n\nShould\n\nCould"},null,null,null,null,null,null,{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Rating]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":""},null,{"$f":"IFERROR(INDEX(QASections_Score[Importance score],MATCH(Table1[[#This Row],[Section]],QASections_Score[Section],0))/SUM(QASections_Score[Importance score])*INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Priority Score]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0))/SUMIF(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Rating]\"),\"<>N/A\",INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Priority Score]\"))*(INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Score before this QA task was undertaken]],Rating_Weight[Rating],0))-INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Expected score after this QA is done]],Rating_Weight[Rating],0)))/10,0)","v":0}]
[null,"Structure & Clarity","Worksheets structure ",{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Outstanding Work]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":"Must\n\nShould\n\nCould"},null,null,null,null,null,null,{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Rating]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":""},null,{"$f":"IFERROR(INDEX(QASections_Score[Importance score],MATCH(Table1[[#This Row],[Section]],QASections_Score[Section],0))/SUM(QASections_Score[Importance score])*INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Priority Score]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0))/SUMIF(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Rating]\"),\"<>N/A\",INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Priority Score]\"))*(INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Score before this QA task was undertaken]],Rating_Weight[Rating],0))-INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Expected score after this QA is done]],Rating_Weight[Rating],0)))/10,0)","v":0}]
[null,"Structure & Clarity","Labels, Units & Conversions",{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Outstanding Work]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":"Must\n\nShould\n\nCould"},null,null,null,null,null,null,{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Rating]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":""},null,{"$f":"IFERROR(INDEX(QASections_Score[Importance score],MATCH(Table1[[#This Row],[Section]],QASections_Score[Section],0))/SUM(QASections_Score[Importance score])*INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Priority Score]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0))/SUMIF(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Rating]\"),\"<>N/A\",INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Priority Score]\"))*(INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Score before this QA task was undertaken]],Rating_Weight[Rating],0))-INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Expected score after this QA is done]],Rating_Weight[Rating],0)))/10,0)","v":0}]
[null,"Structure & Clarity","Formatting",{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Outstanding Work]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":"Must\n\nShould\n\nCould"},null,null,null,null,null,null,{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Rating]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":""},null,{"$f":"IFERROR(INDEX(QASections_Score[Importance score],MATCH(Table1[[#This Row],[Section]],QASections_Score[Section],0))/SUM(QASections_Score[Importance score])*INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Priority Score]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0))/SUMIF(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Rating]\"),\"<>N/A\",INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Priority Score]\"))*(INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Score before this QA task was undertaken]],Rating_Weight[Rating],0))-INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Expected score after this QA is done]],Rating_Weight[Rating],0)))/10,0)","v":0}]
[null,"Structure & Clarity","Workbook Comments",{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Outstanding Work]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":"Must\n\nShould\n\nCould"},null,null,null,null,null,null,{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Rating]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":""},null,{"$f":"IFERROR(INDEX(QASections_Score[Importance score],MATCH(Table1[[#This Row],[Section]],QASections_Score[Section],0))/SUM(QASections_Score[Importance score])*INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Priority Score]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0))/SUMIF(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Rating]\"),\"<>N/A\",INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Priority Score]\"))*(INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Score before this QA task was undertaken]],Rating_Weight[Rating],0))-INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Expected score after this QA is done]],Rating_Weight[Rating],0)))/10,0)","v":0}]
[null,"Structure & Clarity","Code Comments",{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Outstanding Work]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":"Must\n\nShould\n\nCould"},null,null,null,null,null,null,{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Rating]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":""},null,{"$f":"IFERROR(INDEX(QASections_Score[Importance score],MATCH(Table1[[#This Row],[Section]],QASections_Score[Section],0))/SUM(QASections_Score[Importance score])*INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Priority Score]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0))/SUMIF(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Rating]\"),\"<>N/A\",INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Priority Score]\"))*(INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Score before this QA task was undertaken]],Rating_Weight[Rating],0))-INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Expected score after this QA is done]],Rating_Weight[Rating],0)))/10,0)","v":0}]
[null,"Structure & Clarity","Formula clarity & robustness",{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Outstanding Work]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":"Must\n\nShould\n\nCould"},null,null,null,null,null,null,{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Rating]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":""},null,{"$f":"IFERROR(INDEX(QASections_Score[Importance score],MATCH(Table1[[#This Row],[Section]],QASections_Score[Section],0))/SUM(QASections_Score[Importance score])*INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Priority Score]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0))/SUMIF(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Rating]\"),\"<>N/A\",INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Priority Score]\"))*(INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Score before this QA task was undertaken]],Rating_Weight[Rating],0))-INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Expected score after this QA is done]],Rating_Weight[Rating],0)))/10,0)","v":0}]
[null,"Structure & Clarity","Named ranges management",{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Outstanding Work]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":"Must\n\nShould\n\nCould"},null,null,null,null,null,null,{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Rating]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":""},null,{"$f":"IFERROR(INDEX(QASections_Score[Importance score],MATCH(Table1[[#This Row],[Section]],QASections_Score[Section],0))/SUM(QASections_Score[Importance score])*INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Priority Score]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0))/SUMIF(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Rating]\"),\"<>N/A\",INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Priority Score]\"))*(INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Score before this QA task was undertaken]],Rating_Weight[Rating],0))-INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Expected score after this QA is done]],Rating_Weight[Rating],0)))/10,0)","v":0}]
[null,"Verification","Formula correctness",{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Outstanding Work]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":"Must\n\nShould\n\nCould"},null,null,null,null,null,null,{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Rating]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":""},null,{"$f":"IFERROR(INDEX(QASections_Score[Importance score],MATCH(Table1[[#This Row],[Section]],QASections_Score[Section],0))/SUM(QASections_Score[Importance score])*INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Priority Score]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0))/SUMIF(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Rating]\"),\"<>N/A\",INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Priority Score]\"))*(INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Score before this QA task was undertaken]],Rating_Weight[Rating],0))-INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Expected score after this QA is done]],Rating_Weight[Rating],0)))/10,0)","v":0}]
[null,"Verification","Usability Testing",{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Outstanding Work]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":"Must\n\nShould\n\nCould"},null,null,null,null,null,null,{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Rating]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":""},null,{"$f":"IFERROR(INDEX(QASections_Score[Importance score],MATCH(Table1[[#This Row],[Section]],QASections_Score[Section],0))/SUM(QASections_Score[Importance score])*INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Priority Score]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0))/SUMIF(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Rating]\"),\"<>N/A\",INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Priority Score]\"))*(INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Score before this QA task was undertaken]],Rating_Weight[Rating],0))-INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Expected score after this QA is done]],Rating_Weight[Rating],0)))/10,0)","v":0}]
[null,"Verification","Code correctness",{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Outstanding Work]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":"Must\n\nShould\n\nCould"},null,null,null,null,null,null,{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Rating]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":""},null,{"$f":"IFERROR(INDEX(QASections_Score[Importance score],MATCH(Table1[[#This Row],[Section]],QASections_Score[Section],0))/SUM(QASections_Score[Importance score])*INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Priority Score]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0))/SUMIF(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Rating]\"),\"<>N/A\",INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Priority Score]\"))*(INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Score before this QA task was undertaken]],Rating_Weight[Rating],0))-INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Expected score after this QA is done]],Rating_Weight[Rating],0)))/10,0)","v":0}]
[null,"Verification","External links",{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Outstanding Work]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":"Must\n\nShould\n\nCould"},null,null,null,null,null,null,{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Rating]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":""},null,{"$f":"IFERROR(INDEX(QASections_Score[Importance score],MATCH(Table1[[#This Row],[Section]],QASections_Score[Section],0))/SUM(QASections_Score[Importance score])*INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Priority Score]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0))/SUMIF(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Rating]\"),\"<>N/A\",INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Priority Score]\"))*(INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Score before this QA task was undertaken]],Rating_Weight[Rating],0))-INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Expected score after this QA is done]],Rating_Weight[Rating],0)))/10,0)","v":0}]
[null,"Verification","Autochecks & Regression Testing",{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Outstanding Work]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":"Must\n\nShould\n\nCould"},null,null,null,null,null,null,{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Rating]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":""},null,{"$f":"IFERROR(INDEX(QASections_Score[Importance score],MATCH(Table1[[#This Row],[Section]],QASections_Score[Section],0))/SUM(QASections_Score[Importance score])*INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Priority Score]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0))/SUMIF(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Rating]\"),\"<>N/A\",INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Priority Score]\"))*(INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Score before this QA task was undertaken]],Rating_Weight[Rating],0))-INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Expected score after this QA is done]],Rating_Weight[Rating],0)))/10,0)","v":0}]
[null,"Validation","Methodology correctness",{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Outstanding Work]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":"Must\n\nShould\n\nCould"},null,null,null,null,null,null,{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Rating]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":""},null,{"$f":"IFERROR(INDEX(QASections_Score[Importance score],MATCH(Table1[[#This Row],[Section]],QASections_Score[Section],0))/SUM(QASections_Score[Importance score])*INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Priority Score]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0))/SUMIF(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Rating]\"),\"<>N/A\",INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Priority Score]\"))*(INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Score before this QA task was undertaken]],Rating_Weight[Rating],0))-INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Expected score after this QA is done]],Rating_Weight[Rating],0)))/10,0)","v":0}]
[null,"Validation","Comparison with historical data / backcasting / alternative models",{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Outstanding Work]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":"Must\n\nShould\n\nCould"},null,null,null,null,null,null,{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Rating]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":""},null,{"$f":"IFERROR(INDEX(QASections_Score[Importance score],MATCH(Table1[[#This Row],[Section]],QASections_Score[Section],0))/SUM(QASections_Score[Importance score])*INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Priority Score]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0))/SUMIF(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Rating]\"),\"<>N/A\",INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Priority Score]\"))*(INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Score before this QA task was undertaken]],Rating_Weight[Rating],0))-INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Expected score after this QA is done]],Rating_Weight[Rating],0)))/10,0)","v":0}]
[null,"Validation","Uncertainty testing",{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Outstanding Work]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":"Must\n\nShould\n\nCould"},null,null,null,null,null,null,{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Rating]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":""},null,{"$f":"IFERROR(INDEX(QASections_Score[Importance score],MATCH(Table1[[#This Row],[Section]],QASections_Score[Section],0))/SUM(QASections_Score[Importance score])*INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Priority Score]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0))/SUMIF(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Rating]\"),\"<>N/A\",INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Priority Score]\"))*(INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Score before this QA task was undertaken]],Rating_Weight[Rating],0))-INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Expected score after this QA is done]],Rating_Weight[Rating],0)))/10,0)","v":0}]
[null,"Validation","Extreme values testing / model breaking",{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Outstanding Work]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":"Must\n\nShould\n\nCould"},null,null,null,null,null,null,{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Rating]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":""},null,{"$f":"IFERROR(INDEX(QASections_Score[Importance score],MATCH(Table1[[#This Row],[Section]],QASections_Score[Section],0))/SUM(QASections_Score[Importance score])*INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Priority Score]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0))/SUMIF(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Rating]\"),\"<>N/A\",INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Priority Score]\"))*(INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Score before this QA task was undertaken]],Rating_Weight[Rating],0))-INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Expected score after this QA is done]],Rating_Weight[Rating],0)))/10,0)","v":0}]
[null,"Validation","Re-performance testing",{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Outstanding Work]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":"Must\n\nShould\n\nCould"},null,null,null,null,null,null,{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Rating]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":""},null,{"$f":"IFERROR(INDEX(QASections_Score[Importance score],MATCH(Table1[[#This Row],[Section]],QASections_Score[Section],0))/SUM(QASections_Score[Importance score])*INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Priority Score]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0))/SUMIF(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Rating]\"),\"<>N/A\",INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Priority Score]\"))*(INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Score before this QA task was undertaken]],Rating_Weight[Rating],0))-INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Expected score after this QA is done]],Rating_Weight[Rating],0)))/10,0)","v":0}]
[null,"Data & Assumptions","Data",{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Outstanding Work]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":"Must\n\nShould\n\nCould"},null,null,null,null,null,null,{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Rating]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":""},null,{"$f":"IFERROR(INDEX(QASections_Score[Importance score],MATCH(Table1[[#This Row],[Section]],QASections_Score[Section],0))/SUM(QASections_Score[Importance score])*INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Priority Score]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0))/SUMIF(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Rating]\"),\"<>N/A\",INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Priority Score]\"))*(INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Score before this QA task was undertaken]],Rating_Weight[Rating],0))-INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Expected score after this QA is done]],Rating_Weight[Rating],0)))/10,0)","v":0}]
[null,"Data & Assumptions","Data transformation",{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Outstanding Work]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":"Must\n\nShould\n\nCould"},null,null,null,null,null,null,{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Rating]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":""},null,{"$f":"IFERROR(INDEX(QASections_Score[Importance score],MATCH(Table1[[#This Row],[Section]],QASections_Score[Section],0))/SUM(QASections_Score[Importance score])*INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Priority Score]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0))/SUMIF(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Rating]\"),\"<>N/A\",INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Priority Score]\"))*(INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Score before this QA task was undertaken]],Rating_Weight[Rating],0))-INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Expected score after this QA is done]],Rating_Weight[Rating],0)))/10,0)","v":0}]
[null,"Data & Assumptions","Assumptions",{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Outstanding Work]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":"Must\n\nShould\n\nCould"},null,null,null,null,null,null,{"$f":"INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Rating]\"),MATCH(INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Code]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0)),INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"[Task Code]\"),0))&\"\"","v":""},null,{"$f":"IFERROR(INDEX(QASections_Score[Importance score],MATCH(Table1[[#This Row],[Section]],QASections_Score[Section],0))/SUM(QASections_Score[Importance score])*INDEX(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Priority Score]\"),MATCH(Table1[[#This Row],[QA task ]],INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]], \" & \", \"_\") & \"_Score[Task]\"),0))/SUMIF(INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Rating]\"),\"<>N/A\",INDIRECT(SUBSTITUTE(Table1[[#This Row],[Section]],\" & \", \"_\") & \"_Score[Priority Score]\"))*(INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Score before this QA task was undertaken]],Rating_Weight[Rating],0))-INDEX(Rating_Weight[Weight],MATCH(Table1[[#This Row],[Expected score after this QA is done]],Rating_Weight[Rating],0)))/10,0)","v":0}]

{"sheet":"QA history","index":10,"n_rows":5,"n_cols":6,"rows_kept":5,"cut":false,"hidden":false,"merged":[]}
[]
[{"$f":"IF(SecurityClassification=\"OFFICIAL\",\"\",SecurityClassification & \" - \")  & DocumentName & \" QA log - version \" & VersionNumber","v":" -  QA log - version "}]
[]
[]
[null,"Code","Date","QA update","Performed by","QA score"]

{"sheet":"Issues Log","index":11,"n_rows":6,"n_cols":13,"rows_kept":6,"cut":false,"hidden":false,"merged":[]}
[]
[{"$f":"IF(SecurityClassification=\"OFFICIAL\",\"\",SecurityClassification & \" - \")  & DocumentName & \" QA log - version \" & VersionNumber","v":" -  QA log - version "}]
[]
[]
[null,"Model Version","Unique Formula Reference","QA analyst","Worksheet ","Cell/Range","Comments","Priority","Impact","Notes","Date found (use ctrl+;)","Date shared with model owner","Issue has been addressed?"]
[null,null,1]

{"sheet":"Version Log","index":12,"n_rows":23,"n_cols":5,"rows_kept":23,"cut":false,"hidden":false,"merged":[[5,1,5,4]]}
[]
[{"$f":"IF(SecurityClassification=\"OFFICIAL\",\"\",SecurityClassification & \" - \")  & DocumentName & \" QA log - version \" & VersionNumber","v":" -  QA log - version "}]
[]
[]
[]
[null,"This worksheet contains the version log of this document. To be updated only by BEIS's Modelling Integrity Team\n\n"]
[]
[]
[null,"Version number","Date","Author","Description of changes"]
[null,1,"2013-06-01","MoJ","Early version of the log. 6 sections included Scope and Specification, Design and Build, Data and Assumptions, Model Testing, Regression Testing and Documentation, with Regression Testing being of the statistical variety."]
[null,2,"2013-11-01","Modelling Integrity Team","New version finalised after first formal model review by Modelling Integrity Team. Same 6 sections as v1.0 but Regression Testing was modified in order to refer to the testing carried out to ensure unintended changes are not introduced while changing the model. Many tasks within the sections modified/introduced. Additional sheets like the Error Log, Quantitative and Qualitative Databases, Verification & Validation tests added. "]
[null,3,"2014-04-01","Modelling Integrity Team","QA log restructured following further use and feedback into 5 key sections (Documentation, Structure & Clarity, Verification, Validation and Data & Assumptions). Significant changes in the QA requirement of each section to obtain a comprehensive but concise document and to avoid duplications. Most of the columns of the previous versions deleted to make the log less cumbersome. Improved model score system introduced to allow different weightings to be applied to sections and to each QA requirement within the sections. Introduction of a macro to record the details of QA performed and introduction of cell protection procedures. Improved guidance."]
[null,3.1,"2014-06-01","Jon Tecwyn","Minor word changes"]
[null,3.2,"2014-06-01","Ale Arbib","Minor structure changes, added print area and some comments"]
[null,3.3,"2014-07-01","Ale Arbib","Slightly changed the protection mechanism: now formatting rows and columns, colouring and formatting text is allowed. Issues Log sheet is unprotected to avoid bugs due to the use of tables. Added header to show the sheet name when printing"]
[null,3.4,"2014-07-28","Katie Walker","Changed the data and assumptions tasks to refer Readiness, Relevance and Impact ratings"]
[null,3.5,"2014-12-02","Ale Arbib","Added an example for each orange tab, improved the \"Rating guidelines\" column and made some other minor changes."]
[null,3.6,"2015-03-09","Ale Arbib","Added notes on QA proportionality. Minor changes to QA tasks: \"Sensitivity and scenario testing\" broaden to \"Uncertainty testing\", \"QA responsibilities …\" changed to governance to include also folder structure. Regression testing moved from \"Validation\" to \"Verification\" and merged with \"Autochecks and Error Trapping\". Added few new links "]
[null,3.7,"2015-06-18","Ale Arbib","Added the missing rating guidelines to Va2"]
[null,3.8,"2015-10-23","Ale Arbib","Updated links (e.g. link to the new Assumptions log). Re-worded some QA tasks to highlight the importance of prioritising key parts of the model"]
[null,3.9,"2016-05-04","Ale Arbib","Added QA plan sheet"]
[null,4,"2016-06-29","MIT","Changed description of QA tasks and of rating guidelines. Added macro to update QA plan"]
[null,4.0999999999999996,"2016-12-16","Ruth Curran","Adapted this log for external use"]

{"sheet":"Lookups","index":13,"n_rows":29,"n_cols":3,"rows_kept":29,"cut":false,"hidden":false,"merged":[]}
[]
[{"$f":"IF(SecurityClassification=\"OFFICIAL\",\"\",SecurityClassification & \" - \")  & DocumentName & \" QA log - version \" & VersionNumber","v":" -  QA log - version "}]
[]
[]
[null,"Rating","Weight"]
[null,"1) Excellent",0]
[null,"2) Good",1]
[null,"3) Some issues",3]
[null,"4) Needs improvement",7]
[null,"5) Significant issues",10]
[null,"N/A","N/A"]
[]
[null,"Priority","Weight"]
[null,"High",9]
[null,"Medium",3]
[null,"Low",1]
[]
[null,"Worst value",{"$f":"MAX(Rating_Weight[Weight])","v":10}]
[]
[null,"Importance","Weight"]
[null,"Essential",3]
[null,"Important",2]
[null,"Best Practice",1]
[]
[null,"Security Classification"]
[null,"OFFICIAL"]
[null,"OFFICIAL - SENSITIVE"]
[null,"SECRET"]
[null,"TOP SECRET"]