2f:T929,md `
The charts are created using [vega-lite](https://vega.github.io/vega-lite/) and the data processing performed using [arquero](https://uwdata.github.io/arquero/) & [D3](https://d3js.org/). Here are some snapshots from the documentation that I think are what you'll need: ##

[Arquero](https://uwdata.github.io/arquero/)

### In browser To use in the browser, you can load Arquero from a content delivery network: ~~~html ~~~ Arquero will be imported into the aq global object. The default browser bundle does not include the Apache Arrow library. To perform Arrow encoding using toArrow() or binary file loading using loadArrow(), import Apache Arrow first: ~~~html ~~~ Alternatively, you can build and import arquero.min.js from the dist directory, or build your own application bundle. When building custom application bundles for the browser, the module bundler should draw from the browser property of Arquero’s package.json file. For example, if using rollup, pass the browser: true option to the node-resolve plugin. Arquero uses modern JavaScript features, and so will not work with some outdated browsers. To use Arquero with older browsers including Internet Explorer, set up your project with a transpiler such as Babel.

### In Node.js or Application Bundles First install arquero as a dependency, via npm install arquero --save or yarn add arquero. Arquero assumes Node version 12 or higher. Import using CommonJS module syntax: ~~~js const aq = require('arquero'); ~~~ Import using ES module syntax, import all exports into a single object: ~~~js import * as aq from 'arquero'; ~~~ Import using ES module syntax, with targeted imports: ~~~js import { op, table } from 'arquero'; ~~~
### Build Instructions To build and develop Arquero locally: - Clone [https://github.com/uwdata/arquero](https://github.com/uwdata/arquero). - Run yarn to install dependencies for all packages. If you don’t have yarn installed, see https://yarnpkg.com/en/docs/install. - Run yarn test to run test cases, yarn perf to run performance benchmarks, and yarn build to build output files. `30:T5fc,md `
## [Vega-lite](https://github.com/vega/vega-embed)
### Exporting and Publishing a Visualization Once you have visualized your data, perhaps you would like to publish it somewhere else on the web. To save an exported image, you can simply right-click a visualization and select "Save Image As..." from the context menu, assuming the default [canvas](https://developer.mozilla.org/en-US/docs/Web/API/Canvas_API) rendering option is used. To include a Vega-Lite visualization on your own web page, you can use the [vega-embed JavaScript package](https://github.com/vega/vega-embed). Take an exported Vega-Lite JSON string, and incorporate it in a web page that imports vega-embed. Here is a basic HTML template, where the JSON specification for your plot produced by \`chart.toObject()\` should be stored in the \`spec\` JavaScript variable: \`\`\`html

\`\`\` For more information on embedding Vega-Lite, see the [vega-embed documentation](https://github.com/vega/vega-embed). `31:T638,riskTool_top_competitors_listing_count = vl.markBar({width: {band: 0.6}}) .data(clean_data_top_competitors.filter(d => d.top_companies != 'FedEx')) // filter a company out to see legend .title('Top Competitors - Listing Counts WoW') .encode( vl.x().fieldT('date').title('Week') .timeUnit('utcweek'), vl.y().count().title('Listing Counts'), // IMPLEMENTED A POTENTIAL FIX FOR THE LEGEND, SEE HERE: https://appcast.slack.com/archives/C049XRG1C6L/p1674156775939649 vl.color().fieldN('top_companies').title('Top Companies').legend({offset: 50}) // <-- These are the parameters that set colors .scale({ // by adjusting the scale, we can control the parameters domain: company_list, // pass the list of 'top companies' for the legend (i.e. hardcode them) scheme: 'tableau10' // pass the color scheme }), vl.tooltip([ {field: 'week_date', format: 'WK ' + '%U', type: 'temporal', title: 'Week', scale: {type: 'utc'}}, {field: 'top_companies', type: 'nominal', title: 'Company'}, {field: 'count', aggregate: 'count', title: 'Listing Counts', format: ','} ]) ) .config({ view: {stroke: null}, axis: {labelFontWeight: 200, titleFontWeight: 25}, axisY: {domain: null, gridOpacity: 0.5, ticks: false, grid: true, labelPadding: 5, labelFontSize: 8, titlePadding: 10}, axisX: {grid: false, labelPadding: 5, tickCount: 5}, legend: {labelFontWeight: 200, titleFontWeight: 25}, title: {fontWeight: 25, offset: 35} }) .padding({bottom: 50, left: 25, right: 25, top: 50}) .render()32:T448,riskTool_top_competitors_listing_count_json = vl.markBar({width: {band: 0.6}}) .data(clean_data_top_competitors) .title('Top Competitors - Listing Counts WoW') .encode( vl.x().fieldT('date').title('Week') .timeUnit('utcweek'), vl.y().count().title('Listing Counts'), vl.color().fieldN('top_companies').title('Top Companies').legend({offset: 50}), vl.tooltip([ {field: 'week_date', format: 'WK ' + '%U', type: 'temporal', title: 'Week', scale: {type: 'utc'}}, {field: 'top_companies', type: 'nominal', title: 'Company'}, {field: 'count', aggregate: 'count', title: 'Listing Counts', format: ','} ]) ) .config({ view: {stroke: null}, axis: {labelFontWeight: 200, titleFontWeight: 25}, axisY: {domain: null, gridOpacity: 0.5, ticks: false, grid: true, labelPadding: 5, labelFontSize: 8, titlePadding: 10}, axisX: {grid: false, labelPadding: 5, tickCount: 5}, legend: {labelFontWeight: 200, titleFontWeight: 25}, title: {fontWeight: 25, offset: 35} }) .padding({bottom: 50, left: 25, right: 25, top: 50}) .toObject()33:T42c,riskTool_top_competitors_listing_count_json_no_data = vl.markBar({width: {band: 0.6}}) .title('Top Competitors - Listing Counts WoW') .encode( vl.x().fieldT('date').title('Week') .timeUnit('utcweek'), vl.y().count().title('Listing Counts'), vl.color().fieldN('top_companies').title('Top Companies').legend({offset: 50}), vl.tooltip([ {field: 'week_date', format: 'WK ' + '%U', type: 'temporal', title: 'Week', scale: {type: 'utc'}}, {field: 'top_companies', type: 'nominal', title: 'Company'}, {field: 'count', aggregate: 'count', title: 'Listing Counts', format: ','} ]) ) .config({ view: {stroke: null}, axis: {labelFontWeight: 200, titleFontWeight: 25}, axisY: {domain: null, gridOpacity: 0.5, ticks: false, grid: true, labelPadding: 5, labelFontSize: 8, titlePadding: 10}, axisX: {grid: false, labelPadding: 5, tickCount: 5}, legend: {labelFontWeight: 200, titleFontWeight: 25}, title: {fontWeight: 25, offset: 35} }) .padding({bottom: 50, left: 25, right: 25, top: 50}) .toObject()34:T406,riskTool_top_competitors_listing_count_python = vl.markBar({width: {band: 0.75}}) .data(cleaner) .title('Top Competitors - Listing Counts WoW') .encode( vl.x().fieldO('week_num').title('Week'), vl.y().fieldQ('count').title('Listing Counts'), vl.color().fieldN('company').title('Top Companies').legend({offset: 50}), vl.tooltip([ {field: 'week_num',type: 'ordinal', title: 'Week'}, {field: 'company', type: 'nominal', title: 'Company'}, {field: 'count', aggregate: 'count', title: 'Listing Counts', format: ','} ]) ) .config({ view: {stroke: null}, axis: {labelFontWeight: 200, titleFontWeight: 25}, axisY: {domain: null, gridOpacity: 0.5, ticks: false, grid: true, labelPadding: 5, labelFontSize: 8, titlePadding: 10}, axisX: {grid: false, labelPadding: 5, labelOverlap: 'greedy', labelAngle: 0}, legend: {labelFontWeight: 200, titleFontWeight: 25}, title: {fontWeight: 25, offset: 35} }) .padding({bottom: 50, left: 25, right: 25, top: 50}) .render()35:T44e,riskTool_top_competitors_bonus = vl.markSquare({size: 100}) .data(clean_data_bonus) .title('Top Competitors - Bonus in Market') .encode( vl.y().fieldN('top_companies').title(null), vl.x().fieldT('week_date'), vl.color().fieldN('bonus_offering').title('Bonus') .scale({ domain: [true, false], range: ['green', 'lightgray'] }), vl.tooltip([ {field: 'week_date', format: 'WK ' + '%U', type: 'temporal', title: 'Week', scale: {type: 'utc'}}, {field: 'top_companies', type: 'nominal', title: 'Company'}, {field: 'bonus_offering', title: 'Bonus'} ]) ) .config({ view: {stroke: null}, axis: {grid: null, labelFontWeight: 200}, axisX: {domain: false, tickCount: 'week', ticks: false, orient: 'top', format: 'WK ' + '%U', title: null, offset: 2.5, labelPadding: 5, tickOpacity: 0.5}, axisY: {domain: false, labelPadding: 25, ticks: null}, legend: {labelFontWeight: 200, titleFontWeight: 25, offset: 50}, title: {fontWeight: 25, offset: 35} }) .padding({bottom: 50, left: 25, right: 25, top: 50}) .render()36:T455,riskTool_top_competitors_bonus_json = vl.markSquare({size: 100}) .data(clean_data_bonus) .title('Top Competitors - Bonus in Market') .encode( vl.y().fieldN('top_companies').title(null), vl.x().fieldT('week_date'), vl.color().fieldN('bonus_offering').title('Bonus') .scale({ domain: [true, false], range: ['green', 'lightgray'] }), vl.tooltip([ {field: 'week_date', format: 'WK ' + '%U', type: 'temporal', title: 'Week', scale: {type: 'utc'}}, {field: 'top_companies', type: 'nominal', title: 'Company'}, {field: 'bonus_offering', title: 'Bonus'} ]) ) .config({ view: {stroke: null}, axis: {grid: null, labelFontWeight: 200}, axisX: {domain: false, tickCount: 'week', ticks: false, orient: 'top', format: 'WK ' + '%U', title: null, offset: 2.5, labelPadding: 5, tickOpacity: 0.5}, axisY: {domain: false, labelPadding: 25, ticks: null}, legend: {labelFontWeight: 200, titleFontWeight: 25, offset: 50}, title: {fontWeight: 25, offset: 35} }) .padding({bottom: 50, left: 25, right: 25, top: 50}) .toObject()37:T5ce,// This performs all the filtering and transformations required to visualize the data. The first filter involves an array that filters the data to only include our definition of "Top Competitors" (i.e. a hand-picked list) clean_data_top_competitors = aq.from(data.filter(d => all_top_competitors.includes(d.company))) // filter data to include only the companies that are part of our agreed list .filter(d => d.keyword == 'urgent+hire' || d.keyword == 'warehouse') // only include keywords 'urgent hire' and 'warehouse' .derive({date: d => op.parse_date(d.date)}) // parse the date .derive({week_date: aq.escape( d => d3.utcWeek(d.date) )}) // get the date of the week .derive({ top_companies: // group competitors by single name d => op.includes(op.lower(d.company), 'amazon') ? 'Amazon' : op.includes(op.lower(d.company), 'fedex') ? 'FedEx' : op.includes(op.lower(d.company), 'capstone') ? 'Capstone' : op.includes(op.lower(d.company), 'target') ? 'Target' : op.includes(op.lower(d.company), 'uline') ? 'Uline' : op.includes(op.lower(d.company), 'cvs') ? 'CVS' : op.includes(op.lower(d.company), 'walmart') ? 'Walmart' : op.includes(op.lower(d.company), 'sysco') ? 'Sysco' : op.includes(op.lower(d.company), 'ups') || op.includes(op.lower(d.company), 'united parcel service') ? 'UPS' : 'Other' }) .objects() // output data as an array of objects38:T731,// This performs all the filtering and transformations required to visualize the data. The first filter involves an array that filters the data to only include our definition of "Top Competitors" (i.e. a hand-picked list) clean_data_bonus = aq.from(data.filter(d => all_top_competitors.includes(d.company))) .filter(d => d.keyword == 'urgent+hire' || d.keyword == 'warehouse') // only include keywords 'urgent hire' and 'warehouse' .derive({date: d => op.parse_date(d.date)}) // first parse the date .derive({ week_date: aq.escape( d => d3.utcWeek(d.date) ), // get the date of the week bonus_offering: d => op.includes(op.lower(d.job_title), 'bonus'), // check if the job title includes the word 'bonus' }) .derive({ top_companies: // group competitors d => op.includes(op.lower(d.company), 'amazon') ? 'Amazon' : op.includes(op.lower(d.company), 'fedex') ? 'FedEx' : op.includes(op.lower(d.company), 'capstone') ? 'Capstone' : op.includes(op.lower(d.company), 'target') ? 'Target' : op.includes(op.lower(d.company), 'uline') ? 'Uline' : op.includes(op.lower(d.company), 'cvs') ? 'CVS' : op.includes(op.lower(d.company), 'walmart') ? 'Walmart' : op.includes(op.lower(d.company), 'sysco') ? 'Sysco' : op.includes(op.lower(d.company), 'ups') || op.includes(op.lower(d.company), 'united parcel service') ? 'UPS' : 'Other' }) .select('week_date', 'top_companies', 'bonus_offering') // group data by week and remove duplicates .dedupe() .orderby(aq.desc('bonus_offering'), 'week_date') // the 3 lines below are to remove the "false" values if there are "true" ones for a company .dedupe('week_date', 'top_companies') .orderby('week_date') .objects() // output data as an array of objects39:Tc01,// Agreed upon list of what constitutes the "Top Competitors" all_top_competitors = { const fedex = [ "FedEx", "FedEx Ground", "FEDEX GROUND", "FedEx Ground Careers US", "FedEx Ground PH Canada", "FedEx Ground PH US", "FedEx Ground Warehouse", "Fedex ISP" ]; const sc1 = [ "Amazon Warehouse", "Amazon.com", "Amazon Workforce Staffing", "Amazon", "Amazon WFS", "Amazon WFS Canada", "Amazon Warehouse Canada", "Amazon HVH" ]; const ups = [ "UPS", "UNITED PARCEL SERVICES", "United Parcel Service", "UPS Canada", "UPS- United Parcel Service", "United Parcel Service (UPS)", "United Parcel Service UPS" ]; const capstone = [ "Capstone Logistics", "Capstone Logistics LLC", "Capstone Logistics, LLC", "Capstone Logistics, LLC." ]; const target = ["Target"]; const uline = ["Uline"]; const sysco = [ "Sysco", "Sysco - Baltimore", "Sysco - Cincinnati", "Sysco - Denver", "Sysco - Baraboo Local", "Sysco - Boston", "Sysco - Buckhead Meat Northeast", "Sysco - Buckhead Meat of Dallas", "Sysco - Buckhead Meat of Denver", "Sysco - Buckhead Meat of Florida", "Sysco - Buckhead Meat of Houston", "Sysco - Buckhead Meat of San Antonio", "Sysco - Central Pennsylvania", "Sysco - Eastern Wisconsin", "Sysco - Hampton Roads", "Sysco - Metro New York", "Sysco - Nashville", "Sysco - Nashville, TN", "Sysco - Newport Meat Northern California", "Sysco - Newport Meat Southern California", "Sysco - North Star Seafood", "Sysco - Omaha", "Sysco - Palisades Ranch", "Sysco - Syracuse", "Sysco - Sysco Specialty Meats Group", "Sysco - Trinity Seafood", "Sysco Atlantic", "Sysco Canada", "Sysco Canada Inc", "Sysco Canada, Inc", "Sysco Capital Region", "Sysco Central Texas", "Sysco Houston", "Sysco New England", "Sysco Northern New England / Westbrook Maine", "Sysco Ottawa", "Sysco Quebec, division de Sysco Canada", "Sysco Southwestern Ontario", "Sysco Syracuse", "Sysco Tannis Food Distributors", "Sysco Toronto", "Sysco Vancouver", "Sysco Virginia", "Sysco San Francisco", "Sysco SyscoGuest Worldwide" ]; const walmart = [ "Walmart 1742", "Walmart 3422", "Walmart 4189", "Walmart 5476", "Walmart Canada", "DC 7030", "Walmart Distribution Center - 6091 - Henderson, NC", "Walmart Distribution Center 6064", "Walmart Distribution Center 6084", "Walmart Distribution Center 9073", "Walmart Distribution Center Loveland, CO", "Walmart Distribution Center, Irvington, AL 7067", "Walmart Distribution Center- 7010 New Caney, TX", "Walmart Distribution Center-9074-Shepherdsville, KY", "Walmart Store 1489", "Walmart Store 319", "Walmart Store 3857", "Walmart Supercenter #0272", "Walmart distribution center 6088 JCC" ]; return fedex.concat(sc1, ups, capstone, target, uline, sysco, walmart); }3a:[] 2d:{"title":"Risk Tool - Top Competitors Charts","cells":[{"id":0,"value":"md `# Risk Tool - Top Competitors Charts`","pinned":false,"hidden":false,"mode":"ojs"},{"id":1657,"value":"toc({\n headers: 'h1,h2',\n title: 'Implementation',\n skip: ['Risk Tool - Top Competitors Charts', 'Appendix']\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":931,"value":"$2f","pinned":false,"hidden":false,"mode":"ojs"},{"id":1544,"value":"$30","pinned":false,"hidden":false,"mode":"ojs"},{"id":1642,"value":"md `
\n## [D3](https://d3js.org/)\n
\n\nI also use one method within the data processing from the library \"D3\". The page specifies the following:\n\nTo link directly to the latest release, copy this snippet:\n\n~~~html\n \n~~~`","pinned":false,"hidden":false,"mode":"ojs"},{"id":1596,"value":"md `

\n# Code snippets`","pinned":false,"hidden":false,"mode":"ojs"},{"id":1524,"value":"md `
\n*To see the rendered charts, load the .csv file \"Top competitors -wk39 - wk43\"*`","pinned":false,"hidden":false,"mode":"ojs"},{"id":531,"value":"viewof file = Inputs.file({label: \"Load .csv\"})","pinned":false,"hidden":false,"mode":"ojs"},{"id":1044,"value":"md `
\n## Listing counts over time\n
\n\nThe two cells below output either the rendered chart using the \".render()\" method as the last line; or the JSON object using the \"toObject()\" method as the last line.`","pinned":false,"hidden":false,"mode":"ojs"},{"id":608,"value":"$31","pinned":false,"hidden":false,"mode":"ojs"},{"id":1504,"value":"$32","pinned":false,"hidden":false,"mode":"ojs"},{"id":1688,"value":"$33","pinned":true,"hidden":false,"mode":"ojs"},{"id":1706,"value":"md `# Using Chuck's Data`","pinned":false,"hidden":false,"mode":"ojs"},{"id":1709,"value":"viewof file1 = Inputs.file({label: \"Data\"})","pinned":true,"hidden":false,"mode":"ojs"},{"id":1711,"value":"tool_data = file1.json()","pinned":true,"hidden":false,"mode":"ojs"},{"id":1713,"value":"cleaner = tool_data.values","pinned":true,"hidden":false,"mode":"ojs"},{"id":1716,"value":"$34","pinned":true,"hidden":false,"mode":"ojs"},{"id":1721,"value":"riskTool_top_competitors_listing_count_python_json = vl.markBar({width: {band: 0.75}})\n .title('Top Competitors - Listing Counts WoW')\n .encode(\n vl.x().fieldO('week_num').title('Week'),\n vl.y().fieldQ('count').title('Listing Counts'),\n vl.color().fieldN('company').title('Top Companies').legend({offset: 50}),\n vl.tooltip([\n {field: 'week_num', type: 'ordinal', title: 'Week'},\n {field: 'company', type: 'nominal', title: 'Company'},\n {field: 'count', title: 'Listing Counts', format: ','}\n ])\n )\n .config({\n view: {stroke: null},\n axis: {labelFontWeight: 200, titleFontWeight: 25},\n axisY: {domain: null, gridOpacity: 0.5, ticks: false, grid: true, labelPadding: 5, labelFontSize: 8, titlePadding: 10},\n axisX: {grid: false, labelPadding: 5, tickCount: 5, labelAngle: 0, labelOverlap: 'greedy'},\n legend: {labelFontWeight: 200, titleFontWeight: 25},\n title: {fontWeight: 25, offset: 35}\n })\n .padding({bottom: 50, left: 25, right: 25, top: 50})\n .toObject()","pinned":true,"hidden":false,"mode":"ojs"},{"id":1047,"value":"md `
\n## Bonus in market\n
\n\nThe two cells below output either the rendered chart using the \".render()\" method as the last line; or the JSON object using the \"toObject()\" method as the last line.`","pinned":false,"hidden":false,"mode":"ojs"},{"id":628,"value":"$35","pinned":false,"hidden":false,"mode":"ojs"},{"id":1508,"value":"$36","pinned":false,"hidden":false,"mode":"ojs"},{"id":1032,"value":"md `

\n## Data transformations\n
\n\nSQL query used:\n~~~sql\n\nSELECT date, job_title, pay_hourly, company, job_market, job_submarket, search_market, search_submarket, keyword, sponsored\nFROM indeed_master\nWHERE dataset = 'WFS'\nAND date BETWEEN cast('2022-09-25' as date) AND cast('2022-10-29' as date)\n\n~~~\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":1051,"value":"md `### For bar chart: listing counts over time`","pinned":false,"hidden":false,"mode":"ojs"},{"id":560,"value":"$37","pinned":true,"hidden":false,"mode":"ojs"},{"id":1747,"value":"md `### *Legend fix*`","pinned":false,"hidden":false,"mode":"ojs"},{"id":1750,"value":"company_list = aq.from(clean_data_top_competitors)\n .select('top_companies')\n .dedupe()\n .array('top_companies')\n .sort()","pinned":true,"hidden":false,"mode":"ojs"},{"id":1054,"value":"md `### For square table: bonus (true/false) over time`","pinned":false,"hidden":false,"mode":"ojs"},{"id":635,"value":"$38","pinned":true,"hidden":false,"mode":"ojs"},{"id":669,"value":"md `
\n## Competitor lists`","pinned":false,"hidden":false,"mode":"ojs"},{"id":662,"value":"$39","pinned":true,"hidden":false,"mode":"ojs"},{"id":88,"value":"
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n

","pinned":false,"hidden":false,"mode":"html"},{"id":20,"value":"md `## Appendix\n\n_Implementation libraries needed for Observable_`","pinned":false,"hidden":false,"mode":"ojs"},{"id":533,"value":"// Parse the file as a csv(). This is only needed to run in this Observable notebook environment\n\ndata = file.csv()","pinned":true,"hidden":false,"mode":"ojs"},{"id":193,"value":"### *Libraries*","pinned":false,"hidden":false,"mode":"md"},{"id":4,"value":"import {vl} from '@vega/vega-lite-api-v5'","pinned":false,"hidden":false,"mode":"ojs"},{"id":36,"value":"import {uniqueValid} from '@uwdata/data-utilities'","pinned":false,"hidden":false,"mode":"ojs"},{"id":59,"value":"import { aq, op } from '@uwdata/arquero'","pinned":false,"hidden":false,"mode":"ojs"},{"id":1653,"value":"import {toc} from \"@nebrius/indented-toc\"","pinned":false,"hidden":false,"mode":"ojs"}],"files":"$W3a","stdlib":"1","titling":"auto"}