31:[] 2f:{"title":"Code.nasa.gov Data Explorations","cells":[{"id":0,"value":"md`# Code.nasa.gov Data Explorations`","pinned":false,"hidden":false,"mode":"ojs"},{"id":5,"value":"html `

This notebook was created as an alternative way to look at the newest code.nasa.gov projects by leveraging the fact that all code project informatin is stored in a JSON.

`","pinned":false,"hidden":false,"mode":"ojs"},{"id":7,"value":"html `

Websites that contain NASA Open-Source Code

`","pinned":false,"hidden":false,"mode":"ojs"},{"id":9,"value":"html`

There are a couple different places you can find lists of NASA open-source code.

`","pinned":false,"hidden":false,"mode":"ojs"},{"id":12,"value":"html`

The website that attempts to be the larger collection across the whole agency and only lists open-source is Code.NASA.Gov

`","pinned":false,"hidden":false,"mode":"ojs"},{"id":13,"value":"html`

If you work in the federal government or inside NASA and would like to also see some closed or government source only code, you should check out software.nasa.gov as well

`","pinned":false,"hidden":false,"mode":"ojs"},{"id":15,"value":"html `

JSONs that contain information on open-source NASA code

`","pinned":false,"hidden":false,"mode":"ojs"},{"id":25,"value":"html`

This JSON: https://raw.githubusercontent.com/nasa/Open-Source-Catalog/master/code.json is harvested by code.gov. It contains the NASA open-source (and government-source) code projects metadata that you can find on their site. We also harvest this JSON into code.nasa.gov where it is transformed slightly into catalog.json and as raw project data to build code.nasa.gov.

`","pinned":false,"hidden":false,"mode":"ojs"},{"id":28,"value":"html`

Catalog.json is used to build code.nasa.gov. You can find the code repository to build code.nasa.gov here. Please submit an issue if you find any problems.

`","pinned":false,"hidden":false,"mode":"ojs"},{"id":216,"value":"html `

Why have both code.json and catalog.json?

Catalog.json is basically code.json with some formatting changed, A.I. tags added, and only contains open-source code with government-only-source code excluded - which is what we need to build code.nasa.gov

`","pinned":false,"hidden":false,"mode":"ojs"},{"id":330,"value":"header_1 = md `# Basic Statistics`","pinned":true,"hidden":false,"mode":"ojs"},{"id":332,"value":"text_total_tags = html `

The total number of code repositories described on code.nasa.gov as of today is ${codejson.releases.length} Of that total, ${filterCodeJSONBySourceNoButton(codejson.releases,\"openSource\").length} are open source and ${filterCodeJSONBySourceNoButton(codejson.releases,\"governmentWideReuse\").length} are government source, meaning other US government agencies can request them.

`","pinned":false,"hidden":false,"mode":"ojs"},{"id":346,"value":"header_2 = md `# Code released or updated in the last ${past_months} months include:`","pinned":false,"hidden":false,"mode":"ojs"},{"id":469,"value":"viewof past_months = slider({\n min: 0, \n max: 120, \n step: 1, \n value: 6, \n title: \"Months\", \n description: \"Limit Code to Last X Months\"\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":406,"value":"text_recent_fix_look_back = html `

In the last ${past_months} months, ${code_released_since_date.length} code projects have been released. They use primarily the computer languages ${makeListStringWithCommas(getUniqueValesOfSomeKeyFromCodeJSON(code_released_since_date,\"first_language\"))} and have the licenses ${makeListStringWithCommas(getUniqueValesOfSomeKeyFromCodeJSON(code_released_since_date,\"permissions_license\"))}.

`","pinned":false,"hidden":false,"mode":"ojs"},{"id":400,"value":"table_codejson_released_since_X_months_ago = Inputs.Table(code_released_since_date)","pinned":true,"hidden":false,"mode":"ojs"},{"id":456,"value":"text_update_date = html `

The data above is limited to only the past ${past_months} months as set by the slider. It was harvested on ${today}.

`","pinned":false,"hidden":false,"mode":"ojs"},{"id":462,"value":"number_of_months_back = past_months","pinned":false,"hidden":false,"mode":"ojs"},{"id":448,"value":"today = moment()._d","pinned":false,"hidden":false,"mode":"ojs"},{"id":357,"value":"date_at_X_months_ago = moment().subtract(number_of_months_back, 'months'); // 7 months, 7 days and 7 seconds ago","pinned":false,"hidden":false,"mode":"ojs"},{"id":361,"value":"date_six_months_ago = date_at_X_months_ago._d","pinned":false,"hidden":false,"mode":"ojs"},{"id":386,"value":"date_six_months_ago_basic_format = moment(date_six_months_ago).format(\"YYYY-MM-DD\");","pinned":false,"hidden":false,"mode":"ojs"},{"id":392,"value":"function getDateXMonthsAgoInRequiredFormat(code_dataset,number_months){\n var now = moment()\n var six_months_ago = now.subtract(number_months, 'months'); \n var date_six_months_ago = six_months_ago._d\n var date_six_months_ago_basic_format = moment(date_six_months_ago).format(\"YYYY-MM-DD\");\n return date_six_months_ago_basic_format\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":351,"value":"code_released_since_date = filterCodeJSONByDate(codejson,date_six_months_ago_basic_format)","pinned":false,"hidden":false,"mode":"ojs"},{"id":491,"value":"licenses_in_last_x_months = makeListStringWithCommas(licenses_in_last_x_months_array)","pinned":true,"hidden":false,"mode":"ojs"},{"id":496,"value":"licenses_in_last_x_months_array = getUniqueValesOfSomeKeyFromCodeJSON(code_released_since_date,\"permissions_license\")","pinned":true,"hidden":false,"mode":"ojs"},{"id":169,"value":"md `-----------------------------------------`","pinned":false,"hidden":false,"mode":"ojs"},{"id":39,"value":"md `## Getting the JSONs`","pinned":false,"hidden":false,"mode":"ojs"},{"id":31,"value":"codejson = d3.json('https://raw.githubusercontent.com/nasa/Open-Source-Catalog/master/code.json')","pinned":true,"hidden":false,"mode":"ojs"},{"id":32,"value":"catalogjson0 = d3.json('https://raw.githubusercontent.com/nasa/code-nasa-gov/master/data/catalog.json')","pinned":true,"hidden":false,"mode":"ojs"},{"id":248,"value":"table_catalog_json = Inputs.Table(catalogjson0)","pinned":true,"hidden":false,"mode":"ojs"},{"id":309,"value":"table_code_json = Inputs.Table(flatten_code_json(codejson.releases))","pinned":true,"hidden":false,"mode":"ojs"},{"id":314,"value":"code_dot_json_additions = flatten_code_json(codejson.releases)","pinned":true,"hidden":false,"mode":"ojs"},{"id":256,"value":"// Plot.dot(catalogjson0, {x: \"Update_Date\", y: \"Labor_Hours\", title: \"species\"}).plot()","pinned":true,"hidden":false,"mode":"ojs"},{"id":262,"value":"Plot.plot({\n width: width,\n x: {\n label: \"Time\",\n tickFormat: Plot.formatMonth(\"en\", \"short\")\n },\n y: {\n grid: true,\n label: \"Reported Labor Hours of Each Code Project\",\n tickFormat: \"s\", \n },\n marks: [\n Plot.barY(catalogjson0, Plot.groupX({y: \"sum\"}, {x: \"Update_Date\", y: \"Labor_Hours\", fill: \"#4e79a7\"}))\n ]\n});","pinned":false,"hidden":false,"mode":"ojs"},{"id":290,"value":"Plot.rectY(catalogjson0, Plot.binX({y: \"count\"}, {x: \"Labor_Hours\", fill: \"NASA Center\"})).plot()","pinned":false,"hidden":false,"mode":"ojs"},{"id":279,"value":"Plot.dot(catalogjson0, {y: \"NASA Center\", x: \"Update_Date\", stroke: \"NASA Center\"}).plot()","pinned":false,"hidden":false,"mode":"ojs"},{"id":298,"value":"Plot.dot(code_dot_json_additions, {y: \"laborHours\", x: \"date_updated\", stroke: \"organization\"}).plot()","pinned":false,"hidden":false,"mode":"ojs"},{"id":166,"value":"md `-----------------------------------------`","pinned":false,"hidden":false,"mode":"ojs"},{"id":119,"value":"md `## Filtering them to only certain projects by Licensing AND/OR Update Date`","pinned":false,"hidden":false,"mode":"ojs"},{"id":41,"value":"md `### Select Source Type`","pinned":false,"hidden":false,"mode":"ojs"},{"id":85,"value":"viewof source = radio({\n title: 'License Type',\n description: 'Code on code.json is both source type, code on catalog.json is open-source only!',\n options: [\n { label: 'open-source', value: 'openSource' },\n { label: 'government-source', value: 'governmentWideReuse' },\n { label: 'both', value: 'both'},\n ],\n value: 'both'\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":44,"value":"md `### Select Metadata Changed Date`","pinned":false,"hidden":false,"mode":"ojs"},{"id":62,"value":"viewof date_proj_added_after = date({\n // title: \"filter to projects added after date\", \n min: \"2000-01-01\",\n max: \"2060-12-31\",\n value: \"2010-01-15\",\n description: \"This filters code projects to only those after this date\"\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":162,"value":"md `-----------------------------------------`","pinned":false,"hidden":false,"mode":"ojs"},{"id":46,"value":"md `## Results From Filter Selections`","pinned":false,"hidden":false,"mode":"ojs"},{"id":203,"value":"md `### What you must likely want to see is the code.json filtered by the source & date selections above:`","pinned":false,"hidden":false,"mode":"ojs"},{"id":205,"value":"codejson_filter_by_date_and_source","pinned":true,"hidden":false,"mode":"ojs"},{"id":231,"value":"html `

Download the Code Projects Since Date Selected With Selected License Type

`","pinned":false,"hidden":false,"mode":"ojs"},{"id":233,"value":"createJsonDownloadButton({name: 'codejson_filter_by_date_and_source', results: codejson_filter_by_date_and_source}, 'codejson_filter_by_date_and_source.json', 'Download codejson_filter_by_date_and_source as JSON')","pinned":false,"hidden":false,"mode":"ojs"},{"id":243,"value":"html `Other intermediate products`","pinned":false,"hidden":false,"mode":"ojs"},{"id":109,"value":"md `#### code.json filtered by date and/or source`","pinned":false,"hidden":false,"mode":"ojs"},{"id":71,"value":"codejson_array = getReleasesFromCodeJSON(codejson)","pinned":false,"hidden":false,"mode":"ojs"},{"id":98,"value":"codejson_filter_by_source_only = filterCodeJSONBySource(codejson_array,source)","pinned":false,"hidden":false,"mode":"ojs"},{"id":80,"value":"codejson_filter_by_date = filterCodeJSONByDate(codejson,date_proj_added_after)","pinned":false,"hidden":false,"mode":"ojs"},{"id":103,"value":"codejson_filter_by_date_and_source = filterCodeJSONBySource(codejson_filter_by_date,source)","pinned":false,"hidden":false,"mode":"ojs"},{"id":128,"value":"md `#### catalog.json filtered by date and/or source`","pinned":false,"hidden":false,"mode":"ojs"},{"id":130,"value":"catalog_array = catalogjson0","pinned":false,"hidden":false,"mode":"ojs"},{"id":138,"value":"catalog_array_filter_by_source = filterCatalogBySource(catalog_array,source)","pinned":false,"hidden":false,"mode":"ojs"},{"id":144,"value":"catalog_array_filter_by_date = filterCatalogByDate(catalog_array,date_proj_added_after)","pinned":false,"hidden":false,"mode":"ojs"},{"id":150,"value":"catalog_array_filter_by_date_and_source = filterCatalogByDate(catalog_array_filter_by_source,date_proj_added_after)","pinned":false,"hidden":false,"mode":"ojs"},{"id":237,"value":"function createJsonDownloadButton(data, filename = 'data.json', buttonTitle = 'Download JSON') {\n const blob = new Blob([JSON.stringify(data, null, 2)], {type: \"text/json\"});\n return DOM.download(blob, filename, buttonTitle);\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":189,"value":"md `-----------------------------------------`","pinned":false,"hidden":false,"mode":"ojs"},{"id":158,"value":"md `NOTE: Currently, catalog.json is updated from code.json manually when code.nasa.gov is re-built. It is common for code.json to have slightly more repositories. If you're harvesting code projects, you'll most likely want to use the code.json!`","pinned":false,"hidden":false,"mode":"ojs"},{"id":192,"value":"md `Just to re-iterate, catalog.json (and code.nasa.gov) does not have government source code, so it has less code projects than code.json!`","pinned":false,"hidden":false,"mode":"ojs"},{"id":186,"value":"md `-----------------------------------------`","pinned":false,"hidden":false,"mode":"ojs"},{"id":54,"value":"md `## Functions Used`","pinned":false,"hidden":false,"mode":"ojs"},{"id":141,"value":"function filterCatalogByDate(catalog_array,date){\n return catalog_array.filter(catalog_array => catalog_array[\"Update_Date\"] > date);\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":134,"value":"function filterCatalogBySource(catalog_array){\n if (source == \"both\" || source == \"openSource\"){return catalog_array}\n else{\n return []\n }\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":55,"value":"function limitCatalogJsonByDate(catalog){\n var catalog_new = []\n for (var i=0; i< catalog.length; i++){\n // if(new Date(catalog[i][\"Update_Date\"]) <= new Date(date_proj_added_after)){\n if(catalog[i][\"Update_Date\"] >= date_proj_added_after){\n catalog_new.push(catalog[i])\n }\n }\n return catalog_new\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":68,"value":"function getReleasesFromCodeJSON(code_json){\n return code_json[\"releases\"]\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":75,"value":"function filterCodeJSONByDate(codejson,date){\n return codejson_array.filter(codejson_array => codejson_array[\"date\"][\"metadataLastUpdated\"] > date);\n}","pinned":true,"hidden":false,"mode":"ojs"},{"id":93,"value":"function filterCodeJSONBySource(code_json_array,source){\n if (source == \"both\"){return code_json_array}\n else{\n return code_json_array.filter(code_json_array => code_json_array[\"permissions\"][\"usageType\"] == source)\n }\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":339,"value":"function filterCodeJSONBySourceNoButton(code_json_array,source){\n return code_json_array.filter(code_json_array => code_json_array[\"permissions\"][\"usageType\"] == source)\n}","pinned":true,"hidden":false,"mode":"ojs"},{"id":411,"value":"function getUniqueValesOfSomeKeyFromCodeJSON(code_datasets,key_name){\n // function onlyUnique(value, index, self) {\n // return self.indexOf(value) === index;\n // } \n \n function uniqueArray2(arr) {\n var a = [];\n for (var i=0, l=arr.length; i