31:Tea4,table = html`

Description	Detail	Cost
All Resources	Project, Environment and Service Costs are scaled by ${cloud_scaling_factor}	$${parseInt(total_resource_cost)}
	Breakdown: All Resources
per Project Costs	Source Code Repository, Artefact Repository, Container Registry, CI/CD pipline, Security, DNS Resources	$${total_per_project}
Environment Costs	Compute Cluster, SQL, Non Relational, Storage	$${total_per_environment} per environment * ${number_environments} environments
Online Service Costs	Data Visualization Service, Data Set Sharing, Data Analytics	$${total_online}
	Breakdown: Project Costs
Source Code Repository	Costs for online source repository (Gitlab) use for a project team $${cost_sourcecode_per_user} per month * ${projectmonths} months * ${teamsize} team size	$${total_cost_sourcecode}
Artefact Repository	Azure Container Registry $${cost_containerregistry_per_day} per day * ${projectdays(projectmonths)} project days	$${total_cost_containerregistry}
CI CD	GitLab Dev Ops Capability : $${cost_cicd_peruser_permonth} per user per month * ${teamsize} team size* ${projectmonths} months	$${total_cicd}
Key Vault Costs	Storage of sensitive credentials	$${total_key_vault}
Domain Name Costs	Named availablity of any services	$${total_dns}
Certificate Costs	Support private channel into compute services	$${total_certificate}
	Breakdown: per Environment Costs
Compute Costs	A kubernetes ${cluster_nodes} node cloud cluster will be maintained to support data processing.	$${total_vm}
Network Costs	Network Costs are related to GB of data transfer. Price per GB when approximately 1 terabyte off data transfer is $${data_transfer_pergigabyte}. There are ${terabyte_downloaded_permonth} terabytes transferred per month	$${total_network}
Relational	There is ${sql_servers} highly available sql storage required. Relational storage is $${cost_relational_permonth} per month for medium size DB	$${total_relational}
Document Storage Storage	There are ${query_engines} query engines (mongo)..
Storage	There is ${terabyte_storage} terabyes of storage in use	$${total_storage}
Event Storage	There are ${queue_services} queue services (event hub/iot hub) required at $${event_storage_per_month }	$${total_event_storage_per_month}
	Breakdown: Online Services
Visualization	For example using shiny.io $${online_service_monthly_cost} per month for ${projectmonths} months	$${total_visualization_service}
Data Set Sharing	For example using Azure Open Data Sets	$${total_online_shared_data_set}
Total Data Analytics	For example using Azure Data Lake Analytics $${data_analytics_cost} per month for ${projectmonths} months	$${total_data_analytics_cost}

`32:[] 2f:{"title":"Resource Costs Estimate for a Data Science Project","cells":[{"id":0,"value":"md`# Resource Costs Estimate for a Data Science Project`","pinned":false,"hidden":false,"mode":"ojs"},{"id":724,"value":"$31","pinned":false,"hidden":false,"mode":"ojs"},{"id":374,"value":"md`*Note: Costs are exclusive of GST and in AUD dollars*`","pinned":false,"hidden":false,"mode":"ojs"},{"id":14,"value":"md`## Assumptions`","pinned":false,"hidden":false,"mode":"ojs"},{"id":16,"value":"md`The following content lays out expected costs for deployment of a components to support a data science activity`","pinned":false,"hidden":false,"mode":"ojs"},{"id":21,"value":"md`- Solution is developed continuously for ${projectmonths} months which works out to ${cloud_cost_days} days off cloud cost days ([ref](http://www.workingdays.com.au/workingdays_holidays_2019.htm#awd))`","pinned":false,"hidden":false,"mode":"ojs"},{"id":23,"value":"projectmonths=5","pinned":false,"hidden":false,"mode":"ojs"},{"id":77,"value":"md`- There will be ${number_environments} enviroments provisioned as the data science capability can be offline while changes are being applied.`","pinned":false,"hidden":false,"mode":"ojs"},{"id":79,"value":"number_environments=1","pinned":false,"hidden":false,"mode":"ojs"},{"id":107,"value":"md`- Services to be provided from the Australia East Region`","pinned":false,"hidden":false,"mode":"ojs"},{"id":109,"value":"md`- A wildcard DNS record will be obtained such that production and other environments are subdomains and to allow users to easily services/reports/data provided`","pinned":false,"hidden":false,"mode":"ojs"},{"id":114,"value":"md`- Deployment to Azure is estimated Aurecon primarily uses Azure for cloud servies.`","pinned":false,"hidden":false,"mode":"ojs"},{"id":205,"value":"md`- A data science team will be ${teamsize} staff`","pinned":false,"hidden":false,"mode":"ojs"},{"id":202,"value":"teamsize=3","pinned":false,"hidden":false,"mode":"ojs"},{"id":313,"value":"md`- There are storage requirements for the data stored in the solution`","pinned":false,"hidden":false,"mode":"ojs"},{"id":310,"value":"terabyte_downloaded_permonth=2 //The amount of data downloaded/uploaded in the month","pinned":false,"hidden":false,"mode":"ojs"},{"id":342,"value":"md`- There will be an ${terabyte_storage} terabyte of data stored per month`","pinned":false,"hidden":false,"mode":"ojs"},{"id":122,"value":"terabyte_storage=2 ","pinned":false,"hidden":false,"mode":"ojs"},{"id":81,"value":"md`- There is expectation of a ${clouddiscountpercentage}% discount to cloud pricing as the project will not run long enough to get available discounts. Further details [here](https://observablehq.com/@ptomlinsongti/cloud-pricing) `","pinned":false,"hidden":false,"mode":"ojs"},{"id":839,"value":"md`- Costs are scaled up by${cloud_scaling_factor} as a contingency factor`","pinned":false,"hidden":false,"mode":"ojs"},{"id":101,"value":"md`# Derivations`","pinned":false,"hidden":false,"mode":"ojs"},{"id":104,"value":"md`Based on the length of the project the number of days of cloud infrastrature costs would be:`","pinned":false,"hidden":false,"mode":"ojs"},{"id":42,"value":"cloud_cost_days = projectdays(projectmonths)","pinned":false,"hidden":false,"mode":"ojs"},{"id":118,"value":"md`## Compute Environment\n\nA kubernetes cloud cluster will be maintained to support data processing.\n\nA solution is formed via the following components.\n `","pinned":false,"hidden":false,"mode":"ojs"},{"id":127,"value":"cluster_nodes=3 ","pinned":false,"hidden":false,"mode":"ojs"},{"id":128,"value":"domain_names=1","pinned":false,"hidden":false,"mode":"ojs"},{"id":130,"value":"sql_servers=1","pinned":false,"hidden":false,"mode":"ojs"},{"id":132,"value":"query_engines=0","pinned":false,"hidden":false,"mode":"ojs"},{"id":135,"value":"queue_services=1","pinned":false,"hidden":false,"mode":"ojs"},{"id":144,"value":"md`## Cost Breakdown\n\n### Costs per solution $${total_resource_cost}\n\n| Component | Cost |\n| --------- | ---- |\n|costs related to the project | $${total_per_project} |\n|costs related to the environments | $${total_per_environment} * ${number_environments} environments |\n|costs related to the online services | $${total_online_services_rand} |\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":544,"value":"total_resource_cost = parseInt(cloud_scaling_factor*( total_per_project + total_per_environment * number_environments + total_online_services_rand) )","pinned":false,"hidden":false,"mode":"ojs"},{"id":692,"value":"md`### Random variables`","pinned":false,"hidden":false,"mode":"ojs"},{"id":681,"value":"function costComposition(){\n return total_online_services_rand + \n total_per_project_rand + \n total_per_environment_rand \n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":627,"value":"total_online_services_rand={\n for (let i = 0; i < samples; ++i) {\n let expected = total_online\n await Promises.delay(delay);\n yield parseInt(stats.rnorm(1,expected,expected*0.1)[0]);\n }\n yield total_per_project\n}\n","pinned":false,"hidden":false,"mode":"ojs"},{"id":578,"value":"total_per_project_rand={\n for (let i = 0; i < samples; ++i) {\n let expected = total_per_project\n await Promises.delay(delay);\n yield parseInt(stats.rnorm(1,expected,expected*0.1)[0]);\n }\n yield total_per_project\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":581,"value":"total_per_environment_rand={\n for (let i = 0; i < samples; ++i) {\n await Promises.delay(delay);\n yield total_per_environment\n }\n yield total_per_environment\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":776,"value":"md`## Cost Aggregation`","pinned":false,"hidden":false,"mode":"ojs"},{"id":159,"value":"total_per_project = total_cost_sourcecode + total_cost_containerregistry + total_cicd + total_security + total_dns","pinned":false,"hidden":false,"mode":"ojs"},{"id":359,"value":"total_per_environment = total_vm + total_network+total_relational+total_storage+total_event_storage_per_month ","pinned":false,"hidden":false,"mode":"ojs"},{"id":780,"value":"total_online = total_visualization_service+total_online_shared_data_set+total_data_analytics_cost","pinned":false,"hidden":false,"mode":"ojs"},{"id":469,"value":"md`## Costs for online services **$${total_online}**`","pinned":false,"hidden":false,"mode":"ojs"},{"id":500,"value":"md`#### Cost Interactive Data Visualization Service \n\nThe data science activity may use online services to host interactive dashboard developed as part of the project. \n\nPricing for one such service is [ref](https://www.shinyapps.io/)\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":511,"value":"online_service_monthly_cost=39","pinned":false,"hidden":false,"mode":"ojs"},{"id":525,"value":"likely_use_of_online_service=30 //percentage of 100%","pinned":false,"hidden":false,"mode":"ojs"},{"id":632,"value":"total_visualization_service = online_service_monthly_cost*projectmonths","pinned":false,"hidden":false,"mode":"ojs"},{"id":611,"value":"md`#### Cost Data Set Sharing \n\nSome capability to share data set[ref](https://azure.microsoft.com/en-us/services/open-datasets/)\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":613,"value":"total_online_shared_data_set=0","pinned":false,"hidden":false,"mode":"ojs"},{"id":648,"value":"md`#### Cost Data Analytics \n\nSome capability to process data in the cloud[ref](https://azure.microsoft.com/en-us/services/data-lake-analytics/)\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":649,"value":"data_analytics_cost = parseInt(100/0.67)","pinned":false,"hidden":false,"mode":"ojs"},{"id":651,"value":"total_data_analytics_cost=data_analytics_cost*projectmonths","pinned":false,"hidden":false,"mode":"ojs"},{"id":154,"value":"md`## Costs per project **$${total_per_project}**\n\nThere are costs related to establishing the project which are not dependent on the number of environments in use\n\n| Component | Cost |\n| --------- | ---- |\n|cost per source code | $${total_cost_sourcecode} |\n|cost per artefact repository | $${total_artefact_repository} |\n| costs per container registry | $${total_cost_containerregistry} |\n|costs related to CICD | $${total_cicd} |\n| costs for security resources | $${total_security} |\n| costs for DNS resources | $${total_dns} |`","pinned":false,"hidden":false,"mode":"ojs"},{"id":172,"value":"md`#### Costs to maintain Source Code for the team for the project duration **$${total_cost_sourcecode}**`","pinned":false,"hidden":false,"mode":"ojs"},{"id":161,"value":"md`The cost for code repository is ~${cost_sourcecode_per_user} per user per month ([ref](https://github.com/pricing#feature-comparison))`","pinned":false,"hidden":false,"mode":"ojs"},{"id":164,"value":"cost_sourcecode_per_user=10","pinned":false,"hidden":false,"mode":"ojs"},{"id":165,"value":"total_cost_sourcecode = cost_sourcecode_per_user * teamsize * projectmonths","pinned":false,"hidden":false,"mode":"ojs"},{"id":177,"value":"md`#### Costs to maintain code artefacts **$${total_cost_containerregistry}**`","pinned":false,"hidden":false,"mode":"ojs"},{"id":215,"value":"md`It appears that the git source code repositories will included the ability of store built artefacts and there will no additional charge for this([ref](https://help.github.com/en/articles/about-github-package-registry))`","pinned":false,"hidden":false,"mode":"ojs"},{"id":217,"value":"total_artefact_repository=0","pinned":false,"hidden":false,"mode":"ojs"},{"id":170,"value":"md`#### Costs to maintain code artefacts **$${total_cost_containerregistry}**`","pinned":false,"hidden":false,"mode":"ojs"},{"id":179,"value":"md`The costs for soure code afterfacts will be approximately $${cost_containerregistry_per_day} per day ([ref](https://azure.microsoft.com/en-au/pricing/details/container-registry/))`","pinned":false,"hidden":false,"mode":"ojs"},{"id":180,"value":"cost_containerregistry_per_day=1","pinned":false,"hidden":false,"mode":"ojs"},{"id":182,"value":"total_cost_containerregistry = cost_containerregistry_per_day * projectdays(projectmonths)","pinned":false,"hidden":false,"mode":"ojs"},{"id":190,"value":"md`#### Costs for CI/CD pipeline **$${total_cicd}**`","pinned":false,"hidden":false,"mode":"ojs"},{"id":192,"value":"md`$${cost_cicd_peruser_permonth} per user per month [ref](https://about.gitlab.com/pricing/)`","pinned":false,"hidden":false,"mode":"ojs"},{"id":194,"value":"cost_cicd_peruser_permonth=19","pinned":false,"hidden":false,"mode":"ojs"},{"id":199,"value":"total_cicd = cost_cicd_peruser_permonth * teamsize * projectmonths","pinned":false,"hidden":false,"mode":"ojs"},{"id":222,"value":"md`#### Security Costs **$${total_security}**`","pinned":false,"hidden":false,"mode":"ojs"},{"id":234,"value":"md`\n| Component | Cost |\n| --------- | ---- |\n|certificate costs | $${total_certificate} |\n|costs related key vault | $${total_key_vault}|`","pinned":false,"hidden":false,"mode":"ojs"},{"id":236,"value":"total_security = parseInt(total_certificate + total_key_vault)\n","pinned":false,"hidden":false,"mode":"ojs"},{"id":237,"value":"md`- Certificate Costs **$${total_certificate}**`","pinned":false,"hidden":false,"mode":"ojs"},{"id":239,"value":"cost_certificate_peryear=745","pinned":false,"hidden":false,"mode":"ojs"},{"id":241,"value":"total_certificate = parseInt(cost_certificate_peryear * projectmonths/12)","pinned":false,"hidden":false,"mode":"ojs"},{"id":243,"value":"md`- Key Vaults use is likely and do have costs but this is minimal **$${total_key_vault}** ([ref](https://azure.microsoft.com/en-au/pricing/calculator/))`","pinned":false,"hidden":false,"mode":"ojs"},{"id":245,"value":"total_key_vault=100","pinned":false,"hidden":false,"mode":"ojs"},{"id":266,"value":"md`#### DNS Costs **$${total_dns}**`","pinned":false,"hidden":false,"mode":"ojs"},{"id":268,"value":"cost_dns_peryear=20","pinned":false,"hidden":false,"mode":"ojs"},{"id":270,"value":"total_dns=cost_dns_peryear * projectmonths /2","pinned":false,"hidden":false,"mode":"ojs"},{"id":277,"value":"md`### Costs per Environment **$${total_per_environment}**`","pinned":false,"hidden":false,"mode":"ojs"},{"id":278,"value":"md`There are costs directly related to individual environments.`","pinned":false,"hidden":false,"mode":"ojs"},{"id":280,"value":"md`\n| Component | Cost |\n| --------- | ---- |\n|cost per kubernetes | $${total_vm} |\n|cost for relational storage | $${total_relational} |\n|cost for network | $${total_network} |\n|cost for storage | $${total_storage} |\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":282,"value":"cloud_scaling_factor = 1.5 // Assumption is that cloud costs are higher than Azure Costs by this factor ","pinned":false,"hidden":false,"mode":"ojs"},{"id":284,"value":"kubernetes_pernode_permonth=186","pinned":false,"hidden":false,"mode":"ojs"},{"id":286,"value":"total_vm=kubernetes_pernode_permonth * cluster_nodes * projectmonths * (100-clouddiscountpercentage)/100 ","pinned":false,"hidden":false,"mode":"ojs"},{"id":319,"value":"md`#### Network Costs **$${total_network}**`","pinned":false,"hidden":false,"mode":"ojs"},{"id":322,"value":"md`Network Costs are related to GB of data transfer. Price per GB when approximately 1 terabyte off data transfer is $${data_transfer_pergigabyte} `","pinned":false,"hidden":false,"mode":"ojs"},{"id":324,"value":"data_transfer_pergigabyte=171/1000\n","pinned":false,"hidden":false,"mode":"ojs"},{"id":325,"value":"total_network=data_transfer_pergigabyte * terabyte_downloaded_permonth *1000 * projectmonths","pinned":false,"hidden":false,"mode":"ojs"},{"id":293,"value":"md`#### Relational Costs **$${total_relational}**`","pinned":false,"hidden":false,"mode":"ojs"},{"id":294,"value":"md`Relational storage is $${cost_relational_permonth} per month for medium size DB ([ref](https://azure.microsoft.com/en-au/pricing/calculator/))`","pinned":false,"hidden":false,"mode":"ojs"},{"id":296,"value":"cost_relational_permonth=350","pinned":false,"hidden":false,"mode":"ojs"},{"id":298,"value":"total_relational=cost_relational_permonth * projectmonths","pinned":false,"hidden":false,"mode":"ojs"},{"id":304,"value":"md`#### Storage Costs **$${total_storage}**`","pinned":false,"hidden":false,"mode":"ojs"},{"id":305,"value":"md`Storage for ${terabyte_storage} Terabyte would be ${terabyte_storage_permonth} per month [ref](https://azure.microsoft.com/en-au/pricing/calculator/)`","pinned":false,"hidden":false,"mode":"ojs"},{"id":340,"value":"terabyte_storage_permonth=20","pinned":false,"hidden":false,"mode":"ojs"},{"id":307,"value":"total_storage=terabyte_storage *terabyte_storage_permonth * projectmonths","pinned":false,"hidden":false,"mode":"ojs"},{"id":851,"value":"md`### Event Storage\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":859,"value":"\nevent_storage_per_month = 30","pinned":true,"hidden":false,"mode":"ojs"},{"id":854,"value":"total_event_storage_per_month = 30 * projectmonths\n","pinned":false,"hidden":false,"mode":"ojs"},{"id":705,"value":"md`# Variability`","pinned":false,"hidden":false,"mode":"ojs"},{"id":659,"value":"chart","pinned":false,"hidden":false,"mode":"ojs"},{"id":50,"value":"html`

`","pinned":false,"hidden":false,"mode":"ojs"},{"id":54,"value":"md`[This library](https://observablehq.com/@ptomlinsongti/project-relationships) has reusable logic for converting project durations between units`","pinned":false,"hidden":false,"mode":"ojs"},{"id":29,"value":"import { projectdays} from '@ptomlinsongti/project-relationships'","pinned":false,"hidden":false,"mode":"ojs"},{"id":84,"value":"md`[This library](https://observablehq.com/@ptomlinsongti/cloud-pricing) has reusable logic for cloud pricing`","pinned":false,"hidden":false,"mode":"ojs"},{"id":69,"value":"//import {clouddiscountpercentage} from '@ptomlinsongti/cloud-pricing-information'","pinned":false,"hidden":false,"mode":"ojs"},{"id":829,"value":"clouddiscountpercentage=0","pinned":false,"hidden":false,"mode":"ojs"},{"id":381,"value":"import {chart, d3, margin} with {bins,data, x} from \"@mbostock/d3-histogram\"","pinned":false,"hidden":false,"mode":"ojs"},{"id":386,"value":"import { transduceGenerator, concat,mapper } from '@ptomlinsongti/transduce-generated-values-into-array'","pinned":false,"hidden":false,"mode":"ojs"},{"id":389,"value":"computation = (x)=> x","pinned":false,"hidden":false,"mode":"ojs"},{"id":391,"value":"xform = mapper(computation)","pinned":false,"hidden":false,"mode":"ojs"},{"id":398,"value":"t = transduceGenerator(xform,concat,[])","pinned":false,"hidden":false,"mode":"ojs"},{"id":408,"value":"costs = t(costComposition())","pinned":false,"hidden":false,"mode":"ojs"},{"id":406,"value":"data = costs.map((c)=>c)","pinned":false,"hidden":false,"mode":"ojs"},{"id":412,"value":"bins = d3.histogram()\n .domain(x.domain())\n .thresholds(x.ticks(10))\n (data)","pinned":false,"hidden":false,"mode":"ojs"},{"id":415,"value":"x = d3.scaleLinear()\n .domain(d3.extent(data)).nice()\n .range([margin.left, width - margin.right])","pinned":false,"hidden":false,"mode":"ojs"},{"id":426,"value":"delay=100","pinned":false,"hidden":false,"mode":"ojs"},{"id":418,"value":"samples =100","pinned":false,"hidden":false,"mode":"ojs"},{"id":424,"value":"stats = require('statdists')","pinned":false,"hidden":false,"mode":"ojs"}],"files":"$W32","stdlib":"1","titling":"auto"}