31:T1047,criteriaData= d3.csvParse(`Category,Criteria,Description General,Region Presence,Available in Australia East General,Investment,Provider is investing in its capability General,Data Classification,Data and Meta Data can be protected to ensure authorized access. General,Documentation,Provider has accessible documentation description how to make use of available resources. General,Data Sovereignty,Will the provider pass a Data Sovereignty test Vendor,Administrative Overhead,Ease of establishing and maintaining relationship for cloud services by the project. Assumption is that commerical overhead of commerical vendor is a significant issue for the project. Vendor,Financial Viability,Provider is an ongoing financially viable provider not likely to disappear with a 5 year horizon. Vendor,Agency Adoption,Other agencies use the provider. Vendor,Strategy Alignment,There is a broader strategy that directs agencies to use the provider. Vendor,Commercial Scale,The provider has significant scale to be able to provide a reasonable service. Vendor,Maturity,The provider has been providing services to similar customers for a significant period of time. Vendor,Subsidized,Are the cost of services subsidized Commercials,Existing Relationship,The agency has an existing relationship with the provider. Commercials,Price Transparency,The cost of consumption of resource can be easily discovered by the project team to be able to reduce cloud spend where appropriate. Commercials,Value for Money,The provider offers market competitive pricing for services which are not inferior to other providers. Commercials,Service Level Agreement,There is a support channel for the project team to be able to get support within defined timeframes. Commercials,Legal Protection,Use of the provider does not unduly expose the project to legal risk. Commercials,Content Ownership,Any content deployed into the provider belongs to the project. Commercials,No Lock In,The project team are not locked into a long term use of the provider. Compliance,Privay Principles,The provider provider capabilities which assist in ensuring individual privacy is maintained Compliance,ASD Certification,??? Security,Programme of Security Best Practive Enhancements,Provider offers capabilities that assist the project team to implement best practices. This can in the form of proactive monitoring and recommendation for actions that will enhance the security of the solution. Security,Encryption,The provider supports the ability to keep confidential information confidential Security,Logging and Auditing,There is some capability that keeps an audit trail of resource provisioning and deprovisioning Security,Threat Detection,There is some capability that monitors for fraudulent access. Security,Misuse Liability Policy,If the solution is hacked and resources are misused there is a policy that limits the liability to the project. Security,Data Encryption At Rest,The provider has an accessible capability to keep data at rest in encrypted form. Operability,Connectivity,The project team can easily access provider services in a secure way. Operability,Billing Transparency,The provider makes resource use transparent. Technical,Scalable,The provider provides capability to scale a solution either vertically or horizontally Technical,Scripting,There is a CLI to provision and deprovision resources via shell or general purpose language. Technical,Portability,With relative ease the project team can deploy the solution to a different provider. Technical,Identity Management/Ease of Access,The provider provides capability or integration to existing capability that will enable users to get access with relative ease. This could include pre-existing integration to existing directories where users are already expected to exist. Technical,Technology Stack Agnostic,The provider supports both Windows and Linux based technology stacks Technical,Compatibility with solution,"The provider provides resources which the solution requires. For example. VMs, Kubernetes Clusters, Sql DBs,…." Technical,Higher Level Saas,Does the provider provider higher level saas services (or just low level) `)32:T15f4,testData= d3.csvParse(`criteria,test,ease Region Presence,Speak with representative to a confirm regional presence. Validate during SDLC that deployment can target Australia based servers.,0.9 Optionality,Deploy a POC using a provider agnostic standard (such as kubernetes) and confirm that deployment is largely standard kubernetes,0.6 Investment,Speak with representative to a confirm growth plans,0.9 Data Classification,"Speak with representative to a confirm how the provider maintains the Data Classification standard (Unclassified, Protected,…) required by the project.",0.8 Documentation,Request access to provider documentation. Deploy a POC and confirm that most questions can be answered in a self service manner.,0.8 Administrative Overhead,Review sample bills from the provider. Specifically review a POC bill and confirm that charges correlate to services consumed.,0.7 Financial Viability,"Request sales figures, customer numbers, strategic plan and review",0.7 Agency Adoption,Request provider to provider references to other agencies and contact,0.7 Strategy Alignment,Request providers sales staff to provide evidence of alignment to Government strategies,0.8 Strategy Alignment,Literature search to identify relevant Government strategies and evaluate providers,0.7 Commercial Scale,"Request providers to identify customer volumes, evaluate market visibility/advertising of the provider",0.8 Maturity,Deploy POC Application To Providers and compare features available in different providers to appreciate maturity of provider compared to others.,0.5 Existing Relationship,Attempt to incure cost from provider to determine if relationship exists ,0.5 Price Transparency,Operate POC Application for one billing cycle and review resource cost following this,0.6 Value for Money,Review bill for 1 month of utilization of POC on provider,0.6 Value for Money,Review rate card for provider and compare with other providers for the services expected to be used in the project.,0.8 Service Level Agreement,Contact provider staff to get evidence of SLA and its operational effect. Is it visible through the support channel,0.7 Service Level Agreement,Does the provider offer a financially backed SLA product offer for each service that is likely to be used.,0.8 Legal Protection,Review Provider Terms and Conditions with respect to unauthorized access to systems,0.7 Legal Protection,Review Provider Terms and Conditions with respect to unauthorized consumption of resource,0.7 Misuse Liability Policy,Discuss with provider representation how costs related to unauthorized resource use costs will be handled,0.8 Misuse Liability Policy,"Deploy a POC and identify if there are control which allow the project to provide a hard stop on resource consumption. E.g. total VMs, provisioned, total CPU hours limit.",0.6 Content Ownership,Review Provider Terms and Conditions with respect to ownership of content stored in the cloud.,0.7 No Lock In,Are there month to month pricing options available for each service expected to be used.,0.9 Privay Principles,Does the organization provider a privacy policy and is it acceptable to the intended use of the service for the project.,0.8 ASD Certification,Has the cloud provider attained the certification level expected by the project. https://www.cyber.gov.au/irap/asd-certified-cloud-services,0.9 Programme of Security Best Practive Enhancements,"Does the cloud provider demonstrate evidence of a programme of security best practive enhancements, available recommendations, risk rating metrics and are these visible during the SDLC.",0.7 Encryption,Does the provider easily provider support to enabling encryption at rest. This should be validated by enabling the option during a prototype deployment., ,, Logging and Auditing,Is there a policy which describes the options to enable audit and logging in the cloud,0.8 Logging and Auditing,Validate by deploying a POC that logging and auditing is easily realizable on the platform.,0.6 Threat Detection,Review provider documentation for evidence of the type so threats which will be monitored.,0.8 ,, Data Encryption At Rest,"Deploy a POC and confirm there are options to secure data at rest in cloud file, messaging, relational and non-relational storage",0.6 Connectivity,"Deploy a POC and ensure that project team member can access the application, control plan and audting information from project team work locations. ",0.6 Billing Transparency,Deploy a POC and review a monthly bill to confirm there are no unexpected charges and that the charges are in line with understanding.,0.6 Scalable,Deploy a POC following a microservice architecture and validate a service intended to scale horizontally can be easily scaled horizontally.,0.6 Scripting,Deploy a POC and use CLI tools to provision some of the resources in order to validate that difficulties with support for automation.,0.6 Portability,Deploy a POC onto a standard deployment architecture (kubernetes) and confirm that deployment is possible.,0.6 Identity Management/Ease of Access,Set up a project team onto the provider and identify the difficulties with provisioning team members and access.,0.6 Technology Stack Agnostic,Review the service catalogue for the provider and note availability of resources by technology stack,0.9 Compatibility with solution,Deploy POC Application To Provider,0.6 Compatibility with solution,Speak to provider agent about ease of deployment to Provider,0.8 Higher Level Saas,Review the service catalogue and confirm there are high level SaaS offerngs which will simplify the activity required by the project,0.9`)33:[] 2f:{"title":"Cloud Provider Programme Choice Due Diligence","cells":[{"id":0,"value":"md`# Cloud Provider Programme Choice Due Diligence`","pinned":false,"hidden":false,"mode":"ojs"},{"id":320,"value":"html`

Work in progress

`","pinned":false,"hidden":false,"mode":"ojs"},{"id":5,"value":"{\n drawHtmlTable(providerListTable,providerProcessedData,(col)=>col==='name')\n return md`A project needs to evaluate an appropriate choice of cloud provider for a solution.\n\n There several provider choices`\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":6,"value":" providerListTable = html`\n

`","pinned":false,"hidden":false,"mode":"ojs"},{"id":32,"value":"md`### Procedure to Evaluate`","pinned":false,"hidden":false,"mode":"ojs"},{"id":199,"value":"{\n drawHtmlTable(providerTable,providerProcessedData)\n return md`##### Initial Estimate\nWe would start an evaluation by collaboratively forming a collective team guess (a prior probability) for each provider that the cloud provider is a suitable choice`\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":192,"value":" providerTable = html`\n

`","pinned":false,"hidden":false,"mode":"ojs"},{"id":535,"value":"md`##### Goal: Criteria`","pinned":false,"hidden":false,"mode":"ojs"},{"id":541,"value":"{\n drawHtmlTable(criteriaTable,criteriaData)\n return md`The goal is that we can identify independent critera which will inform our perception of any provider. These criteria need to be combinable into a measure of the suitability of the provider. The measures of any one provider should be comparable to the measures for other providers.`\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":416,"value":"{ \n let provider = providerProcessedData[0]\n let p = {name: provider.name, children: []}\n criteriaData.forEach((criteria)=>{\n let c = {name: criteria.Criteria,children: []}\n p.children.push(c)\n })\n\n return chart(p)\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":558,"value":"md`The set of criteria to be consider can include the following`","pinned":false,"hidden":false,"mode":"ojs"},{"id":141,"value":" criteriaTable = html`\n

`","pinned":false,"hidden":false,"mode":"ojs"},{"id":523,"value":"md`##### Criteria Likelihoods`","pinned":false,"hidden":false,"mode":"ojs"},{"id":58,"value":"md`For each criteria we would need to identify the probability that a suitable provider would exhibit the criteria in the form of a likelihood table. Below is an example of Compatibility (Criteria) related to the appropriateness of the Azure (provider)`","pinned":false,"hidden":false,"mode":"ojs"},{"id":236,"value":" drawLikelihood(likelihood_processed_data[0]);","pinned":false,"hidden":false,"mode":"ojs"},{"id":510,"value":"md`**Note: Read this tables as: *If there were 100 different projects and we know that .6 should be appropriate for Azure (our prior) then how do we think evidence of Compatibility relates to the appropriateness of Azure*`","pinned":false,"hidden":false,"mode":"ojs"},{"id":515,"value":"md`The **Accuracy** of the likelihood table (the number of True Positive and True Negatives to the total projects) gives us an orderable measure for how significant the criteria is in making a decision.`","pinned":false,"hidden":false,"mode":"ojs"},{"id":518,"value":"md`We would order the criteria in terms of accuracy and then continue by considering steps below for those highest accuracy criteria`","pinned":false,"hidden":false,"mode":"ojs"},{"id":528,"value":"md`##### Using Known Criteria`","pinned":false,"hidden":false,"mode":"ojs"},{"id":257,"value":"md`If we had evidence to support a belief that the provider meets the criteria, we can use the likelihoods to update our beliefs that provider appropraite. \n\nBelow is an example where if we have evidence that Compatibility (criteria) is met and we update our estimate for the suitability for Azure (provider).`","pinned":false,"hidden":false,"mode":"ojs"},{"id":268,"value":"posterior = computePosterior(providerProcessedData[0],likelihood_processed_data[0])","pinned":false,"hidden":false,"mode":"ojs"},{"id":354,"value":"md`The estimate is updated from ${providerProcessedData[0].positive} to ${posterior.posterior}`","pinned":false,"hidden":false,"mode":"ojs"},{"id":299,"value":"md`##### Goal Combined Criteria Estimates\n\n\nThe goal is to be able to have a likelihood relationship for each criteria.\nEvaluate if there is evidence that each criteria is met by the provider (or not)\nThen update the belief that the provider is suitable using the likelihood tables.\n\nBelow is an example calculation updating the belief that a particular provider is appropriate given evidence for a couple criteria being satisifed`","pinned":false,"hidden":false,"mode":"ojs"},{"id":463,"value":"posteriorFromMany = updateBelief({prior: providerProcessedData[0],likelihoods})","pinned":false,"hidden":false,"mode":"ojs"},{"id":474,"value":"md`The computed posterior is ${posteriorFromMany.positive} after considering \n\n**${posteriorFromMany.description}**\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":602,"value":"md`The prior belief was updated consider these 2 likelihood estimates`","pinned":false,"hidden":false,"mode":"ojs"},{"id":597,"value":" drawLikelihood(likelihood_processed_data[0]);","pinned":true,"hidden":false,"mode":"ojs"},{"id":599,"value":" drawLikelihood(likelihood_processed_data[1]);","pinned":false,"hidden":false,"mode":"ojs"},{"id":303,"value":"md`##### Gaining evidence to support Criteria`","pinned":false,"hidden":false,"mode":"ojs"},{"id":43,"value":"{\n drawHtmlTable(testTable,testData)\n return md`In order to justify that our criteria evaluations are valid, the suggestion is that we would identify executable 'tests' which provides insight and evidence for the performance of that provider in terms of the criteria. Below are proposed tests and an estimate of their ease of implementation.`\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":612,"value":" testTable = html`\n

`","pinned":false,"hidden":false,"mode":"ojs"},{"id":572,"value":"md`##### Goal Combine Test Evidence\n\nThe goal in executing tests is to use test data to updated our belief that the provider meets the criteria.\nThen recursively use the set of criteria to update the estimate of the provider.\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":482,"value":"{ \n let provider = providerProcessedData[0]\n let p = {name: provider.name, children: []}\n criteriaData.forEach((criteria)=>{\n let c = {name: criteria.Criteria,children: []}\n p.children.push(c)\n testData.forEach((test)=>{\n if(test.criteria===criteria.Criteria){\n let t = {name: test.test}\n c.children.push(t)\n }\n })\n })\n\n return chart(p)\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":67,"value":"md`As tests are identified , we would update an ordering of the the tests by their influence onto the suitability choice and their ease of execution.`","pinned":false,"hidden":false,"mode":"ojs"},{"id":93,"value":"md`We would start executing tests as they are identified and move to the top of the ordering list.`","pinned":false,"hidden":false,"mode":"ojs"},{"id":85,"value":"md`Note: *The risk profile for the customer will determine how many of the tests are executed. A higher risk tolerance can mean fewer tests are considered.*`","pinned":false,"hidden":false,"mode":"ojs"},{"id":88,"value":"md`As each test result becomes available we update the prior probabilities the the providers are appropriate.`","pinned":false,"hidden":false,"mode":"ojs"},{"id":367,"value":"md`**Note** that we can perform this procedure with anonymized providers and test results. This would allow all stakeholders to participate in the estimation logic without political considerations.`","pinned":false,"hidden":false,"mode":"ojs"},{"id":185,"value":"md`---`","pinned":false,"hidden":false,"mode":"ojs"},{"id":105,"value":"providerData = d3.csvParse(`name,positive\nAzure,0.6\nGCP,0.4\nAWS,0.4\nMacquarieGov,0.4\nCSIRO,0.6\nGBRMPA,0.4\nIMOS,0.2\nAIMS,0.2\nBOM,0.2\nOGBR,0.2`)","pinned":false,"hidden":false,"mode":"ojs"},{"id":381,"value":"$31","pinned":false,"hidden":false,"mode":"ojs"},{"id":402,"value":"providerProcessedData = providerData.map( (provider)=>{\n let result = Object.assign({},provider)\n //result.negative = 1 - provider.positive\n return result\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":138,"value":"import {drawHtmlTable} from '@ptomlinsongti/library-to-draw-a-table'","pinned":false,"hidden":false,"mode":"ojs"},{"id":108,"value":"likelihoods = ([{\n description: \"When compability off the solution is considered important azure is a goodd choice\",\n predicate: \"Compatibility\",\n actual: \"Azure\",\n TP: 60,\n TN: 10,\n FP: 30,\n },{\n description: \"When overhead is considered problematic azure is not the provider of choice\",\n predicate: \"Administrative Overhead\",\n actual: \"Azure\",\n TP: 59,\n TN: 10,\n FP: 30,\n },{\n description: \"Unrelated likelihood\",\n predicate: \"Foo\",\n actual: \"Bar\",\n TP: 59,\n TN: 10,\n FP: 30,\n }]) ","pinned":false,"hidden":false,"mode":"ojs"},{"id":246,"value":"likelihood_processed_data = likelihoods.map( (likelihood)=>{\n return {\n description: likelihood.description,\n predicate: likelihood.predicate,\n actual: likelihood.actual,\n TP: likelihood.TP,\n TN: likelihood.TN,\n FP: likelihood.FP\n }\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":307,"value":"$32","pinned":false,"hidden":false,"mode":"ojs"},{"id":227,"value":"import {drawLikelihood} from '@ptomlinsongti/library-to-draw-a-likelihood-table'\n","pinned":false,"hidden":false,"mode":"ojs"},{"id":279,"value":"import { updateBelief,computePosterior} from '@ptomlinsongti/deriving-posterior-probability'","pinned":false,"hidden":false,"mode":"ojs"},{"id":382,"value":"d3 = require(\"d3@5\")","pinned":true,"hidden":false,"mode":"ojs"},{"id":412,"value":"import {chart,tree} from '@ptomlinsongti/library-to-draw-a-tree'","pinned":false,"hidden":false,"mode":"ojs"}],"files":"$W33","stdlib":"1","titling":"auto"}