2f:T5ef,md`# Anonymization Puzzle The goal here is to compute some simple statistics on a dataset of numbers belonging to users. However, the challenge is that there is a simple anonymization scheme the data must pass through. It can be summarized as: 1. Each user must be processed first independently. The output may not duplicate any row. 2. The rows across all users must be grouped and each group counted. 3. Then each group with 5 or fewer rows must be discarded.

The challenge is to figure out how to process each user and how to post-proccess the anonymized groups to get the most accurate statistics possible. I think this challenge is sort of the inverse of [Anscombe's Quartet](https://en.wikipedia.org/wiki/Anscombe%27s_quartet) in the sense that one has to fudge the data sufficiently that the statistics appear the same, even though the datapoints are no longer distuinguishable. Note that this is an interactive notebook, you can click next to any cell to reveal the code powering it. ## Data preparation First we pre-process the data to only get the ticket prices for each user, since the other data is just chaff. I did this with the following terminal command: cat data/*.json | jq -s '[.[].purchases | map(select(.type == "airline").amount)]' Which produces the following (click around to expand): `30:T1766,data = [ [ 150 ], [ 150 ], [ 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000 ], [ 10000, 10000, 10000, 10000, 10000 ], [ 10000, 10000, 10000, 10000, 10000 ], [ 10000, 10000, 10000, 10000, 10000 ], [ 10000, 10000, 10000, 10000, 10000 ], [], [ 9000, 9000, 9000 ], [], [ 150 ], [ 150 ], [ 150 ], [ 150 ], [ 150 ], [ 150 ], [ 150 ], [ 150 ] ] 31:T9ee,md`## Per-user Processing Next, it's time to define the per-user processing program. This is challenging, because we need to be clever to output values that will survive anonymization - which means that we need at least another ${N} users to agree on the same values! However, it is typically the case that outliers will dominate the statistics we've been asked to produce. There are a few insights that I would like to point out: - Users that have bought no tickets effectively do not count (as they don't affect any of the statistics). We can remove them from consideration. The exception to this is that if there are not enough users with tickets (${tex`2N+1`}) for the other techniques to work, it may be useful to include them. I've not done that here for simplicity. - We can know exactly how many users (with tickets) there are by simply having each user emit the \`['user']\` tag. Since it's exacly the same for each user, we know it will survive anonymization better than anything else. - We need a way to group arbitrary numbers in a way that multiple users can agree on them. I've opted for a simple solution here: we will create a number of buckets that the numbers can possibly be in. We can then emit a value for each bucket along with a boolean wether the user's number falls into that bucket. In order to create these buckets, we need a lower bound, an upper bound and a number of buckets to create (\`resolution\`). These become parameters to the programs. Note: We create the buckets by linearly dividing the range. I've experimented with logarithmic subdivisions, but I dislike the additional assumptions about the data this entails. - We can find out positive facts about groups of users who are unusual (i.e. their count is ${tex`N`} or fewer), if we have all other users deny that fact. We can then subtract the count of denials from the total users to recover the true count. For example, if 15 users claim that their average ticket price **wasn't** in the range 8,500-9,500 and there were 16 users in total, we can deduce that there was a single user whose average ticket price **was** in that range, even if the data from that single user claiming that fact gets erased in anonymization. Note: for this to work, we need at least ${tex`2N+1`} users with data, since we guarantee that for each question there will be more than ${tex`N`} of either positive or negative answers. Using this technique, we will have each user emit the sum, average, count, minimum and maximum of their ticket prices. `32:T402,html`

${md`## Parameters We need to choose appropriate bounds to create the buckets. The program will crash if the buckets don't actually fit the data. `}

Results after anonymization

	Result	Error
Minimum	${solution.min.toFixed( 1 )}	Δ${( solution.min - correct.min ).toFixed(1)}
Maximum	${solution.max.toFixed( 1 )}	Δ${( solution.max - correct.max ).toFixed(1)}
Average	${solution.avg.toFixed( 1 )}	Δ${( solution.avg - correct.avg ).toFixed(1)}
Median	${solution.median.toFixed( 1 )}	Δ${( solution.median - correct.median ).toFixed(1)}

`33:T57f,Range = { const Range = (min, max) => { const center = min + (max - min) / 2; return { min, max, center, valueOf() { return center; }, toFixed(num) { // This is used for formatting the range return `${center.toFixed(num)} ±${((max - min) / 2).toFixed(num)}`; }, add(otherRange) { return Range(min + otherRange.min, max + otherRange.max); }, divBy(otherRange) { // Additional operations always increase uncertainty, note the switched values: return Range(min / otherRange.max, max / otherRange.min); }, scale(num) { // multiply the range by a scalar number return Range(min * num, max * num); }, mult(otherRange) { return Range(min * otherRange.min, max * otherRange.max); }, contains(otherRange) { return otherRange.min >= min && otherRange.max <= max; }, intersects(otherRange) { return otherRange.min <= max && min <= otherRange.max; }, has(value) { return value >= min && value <= max; } }; }; Range.zero = Range(0, 0); Range.min = arr => arr.reduce((m, a) => (m < a.valueOf() ? m : a), Range(Infinity, Infinity)); Range.max = arr => arr.reduce( (m, a) => (m > a.valueOf() ? m : a), Range(-Infinity, -Infinity) ); return Range; }34:T405,postProcess = data => { const totalUsers = data.filter(([[tag]]) => tag === 'user')[0][1]; const { results } = data.reduce( ({ results, seenRanges }, [[stat, min, max, positive], count]) => { if ( stat !== 'user' && (positive || count !== totalUsers) && !seenRanges[stat].has(min) // It can happen that both a positive and negative group survives anonymization. In this case we have to keep track of this and only include it once in the output. ) { seenRanges[stat].add(min); results[stat].push({ count: positive ? count : totalUsers - count, range: Range(min, max) }); } return { results, seenRanges }; }, { results: { totalUsers, min: [], max: [], avg: [], count: [], sum: [] }, seenRanges: { min: new Set(), max: new Set(), avg: new Set(), count: new Set(), sum: new Set() } } ); return results; }35:T40d,md`### Median So far we have not covered how to compute the median. This is by far the trickiest of the statistics, since from a per-user perspective it is impossible to tell which of the values will be the median. Furthermore, after the anonymization phase is complete, it is very difficult to attach more than a single number to the same user. All of this makes computing an exact median, which requires all the values to be sorted and the middle picked extremely tricky. I have opted here to compute a reasonable estimate by 1. Figuring out for each user a tuple of (average ticket price, ticket count). 2. Essentially making the right amout of tickets for each user at their average ticket price. 3. Computing the median of these. Since in the sample dataset each user only buys tickets at a single price, this ends up being equivalent to the median, but I believe in general this is not a totally miserable approximation to a median. Crucially though, we do loose the property that the median actually occurs in the dataset. `36:T8a4,reconstructUsers = stats => { // 1. Create arrays of ranges const sums = stats.sum.reduce(pushN, []).map((value, id) => ({ value, id })); const counts = stats.count.reduce(pushN, []); const avgs = stats.avg.reduce(pushN, []); // 2. Combine these into candidate solutions const candidates = avgs.map(avg => { const permutations = filterMap(counts, (count, id) => { const sum = avg.mult(count); // Let's see if there are sums that could explain this combination? const possibleSums = sums.filter(sum2 => sum2.value.intersects(sum)); if (possibleSums.length > 0) { const error = sums.reduce( (err, rsum) => Math.min(err, Math.abs(rsum.value.valueOf() - sum.valueOf())), Infinity ); return { sum, avg, count, error, id, possibleSums }; } }); return permutations.sort( (a, b) => a.possibleSums.length === b.possibleSums.length || a.error - b.error ); }); // Sort the candidate solutions by the number of possible assignments candidates.sort((a, b) => a.length - b.length); // Now recursively figure out a consistent set of assignments const assign = ( [candidate, ...candidates], seen = new Set(), solutions = [] ) => { if (!candidate) { // Base case return solutions; } // We've ran out of options here, backtrack. if (candidate.length === 0) return false; const bestSolution = candidate[0]; const possibleSums = bestSolution.possibleSums.filter(ps => !seen.has(ps)); const newSeen = new Set(seen); if (possibleSums.length === 0) { // This assignment has already been used in this exploration, backtrack return false; } else { newSeen.add(possibleSums[0].id); } // recurse const result = assign( candidates.map(c => c.filter(({ id }) => id !== bestSolution.id)), newSeen, [bestSolution, ...solutions] ); if (result) return result; else { // Backtracked. Remove the current assignment, then try again, candidate.shift(); return assign([candidate, ...candidates], seen, solutions); } }; return assign(candidates); }37:T8bd,manualTestSuite( 'Anonymization Puzzle', ({ describe, it, xit, expect, jasmine, beforeEach }) => { beforeEach(() => setupFastCheck(jasmine)); it('always contains the real point in the uncertainty bounds', () => { expect(data => { const anonymized = anonymize(data.map(processUserRecord)); const result = aggregate(anonymized); const solution = computeStatsWithoutAnonymization(data); return ( result.min.has(solution.min) && result.max.has(solution.max) && result.avg.has(solution.avg) && // We don't have a great check for median, but let's just check it's reasonable result.median >= result.min.min && result.median <= result.max.max ); }).forAll( fc.array(fc.array(fc.integer(minPrice, maxPrice), 1, maxCount), 11, 20) ); }); describe('Anonymization', () => { it('eliminates values that are present <= 5', () => { const result = anonymize([['a'], ['a'], ['a'], ['a'], ['a']]); expect(result).toEqual([]); }); it('keeps values that are present > 5', () => { const result = anonymize([['a'], ['a'], ['a'], ['a'], ['a'], ['a']]); expect(result).toEqual([['a', 6]]); }); it('never produces a count less than 6', () => { expect(data => { const result = anonymize(data); return result.every(([_, count]) => count > 5); }).forAll(fc.array(fc.array(fc.constantFrom('a', 'b', 'c', 'd')))); }); }); describe('computeStatsWithoutAnonymization', () => { it('computes somewhat reasonable values', () => { expect(data => { const results = computeStatsWithoutAnonymization(data); return results == null || results.min <= results.max; }).forAll(fc.array(fc.array(fc.float()))); }); }); describe('per user processing', () => { it('never creates duplicate rows', () => { expect(user => { const processed = processUserRecord(user); return JSON.stringify(processed) === JSON.stringify(uniq(processed)); }).forAll(fc.array(fc.integer(minPrice, maxPrice), 0, maxCount)); }); }); } )38:[] 2d:{"title":"Anonymization Puzzle","cells":[{"id":0,"value":"$2f","pinned":false,"hidden":false,"mode":"ojs"},{"id":4,"value":"$30","pinned":false,"hidden":false,"mode":"ojs"},{"id":716,"value":"N = 5","pinned":false,"hidden":false,"mode":"ojs"},{"id":721,"value":"md`${tex`N = ${N}`} represents the number of other users to agree with a user in order for them to survive anonymization.`","pinned":false,"hidden":false,"mode":"ojs"},{"id":12,"value":"$31","pinned":false,"hidden":false,"mode":"ojs"},{"id":373,"value":"$32","pinned":false,"hidden":false,"mode":"ojs"},{"id":253,"value":"viewof maxPrice = slider({\n min: 0,\n max: 100000,\n value: 50000,\n step: 1,\n format: '$,',\n title: 'Maximum Ticket Price',\n description: 'What is the most a single person could pay for a single ticket?'\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":257,"value":"viewof minPrice = slider({\n min: 0,\n max: 1000,\n value: 0,\n step: 1,\n format: '$,',\n title: 'Minimum Ticket Price',\n description:\n 'What is the least a single person could pay for a single ticket?'\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":260,"value":"viewof maxCount = slider({\n min: 1,\n max: 2000,\n value: 1000,\n step: 1,\n format: ',',\n title: 'Maximum Ticket Count',\n description: 'What is the most tickets we expect any single user could buy?'\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":268,"value":"viewof resolution = slider({\n min: 100,\n max: 10000,\n value: 5000,\n step: 1,\n format: ',',\n title: 'Resolution',\n description:\n 'This affects how fine grained the ranges we will use for estimation will be. Generally the higher this value will be, the more time and space this program will require. A larger range of prices and larger ticket counts will need a higher resolution to stay accurate.',\n submit: true\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":15,"value":"processUserRecord = tickets => {\n if (tickets.length === 0) {\n return [];\n }\n\n const sum = tickets.reduce((sum, val) => sum + val, 0);\n\n const ranges = [['user']];\n\n const addRanges = (key, value, min = minPrice, max = maxPrice) => {\n const inc = Math.max(1, Math.floor((max - min) / resolution));\n for (let start = min; start <= max; start += inc) {\n const end = start + inc;\n ranges.push([key, start, end, value >= start && value < end]);\n }\n };\n\n addRanges('sum', sum, minPrice, maxPrice * maxCount);\n\n addRanges('count', tickets.length, 1, maxCount);\n\n addRanges('avg', sum / tickets.length);\n\n addRanges('min', Math.min(...tickets));\n\n addRanges('max', Math.max(...tickets));\n\n return ranges;\n}","pinned":true,"hidden":false,"mode":"ojs"},{"id":19,"value":"md`As an example, here's the output of running it on a user (userId=4):`","pinned":false,"hidden":false,"mode":"ojs"},{"id":22,"value":"processUserRecord(data[4])","pinned":false,"hidden":false,"mode":"ojs"},{"id":29,"value":"md`## Anonymization\n\nNow, let's define the anonymization scheme as required. There is some mucking about in the algorithm since the native \\`Map\\` datastructure doesn't support other equality than reference equality. Hence we use serialized values as keys.`","pinned":false,"hidden":false,"mode":"ojs"},{"id":37,"value":"anonymize = users =>\n [\n ...users\n .reduce((groups, userData) => {\n userData.forEach(val => {\n const value = JSON.stringify(val);\n if (groups.has(value)) {\n const [fetched, count] = groups.get(value);\n groups.set(value, [fetched, count + 1]);\n } else {\n groups.set(value, [val, 1]);\n }\n });\n return groups;\n }, new Map())\n .values()\n ].filter(([_, count]) => count > N)","pinned":true,"hidden":false,"mode":"ojs"},{"id":76,"value":"md`## Aggregation\n\nFirst, let's compute the stats quickly without anonymization just to verify that we've imported the data correctly.\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":80,"value":"computeStatsWithoutAnonymization = data => {\n const flat = data.flat();\n if (flat.length === 0) return null;\n return {\n min: Math.min(...flat),\n max: Math.max(...flat),\n avg: flat.reduce((a, b) => a + b, 0) / flat.length,\n median:\n flat.length % 2 === 1\n ? flat[(flat.length - 1) / 2]\n : (flat[flat.length / 2 - 1] + flat[flat.length / 2]) / 2\n };\n}","pinned":true,"hidden":false,"mode":"ojs"},{"id":83,"value":"correct = computeStatsWithoutAnonymization(data)","pinned":false,"hidden":false,"mode":"ojs"},{"id":162,"value":"md`OK, now to the real deal here. How do we actually produce any reasonable stats?\n\nFirst, let's define a data structure to help us model the problem. Given how we are getting the data from anonymization, we never really get exact values, but rather these \"buckets\", or as we can think of them Ranges of values. Essentially we know that the real value is inside the interval, but there is uncertainty what the value is exactly. The good thing is we can define a whole bunch of useful operations with these:`","pinned":false,"hidden":false,"mode":"ojs"},{"id":483,"value":"$33","pinned":true,"hidden":false,"mode":"ojs"},{"id":753,"value":"md`Next, we need to get the groups of values that survived anonymization into a sensible datastructure, that we can compute with. To do this we:\n\n1. Count the total number of users.\n2. For each value that survived, we create a range from it and assign it a count:\n - If the range includes the value, then it is the count of the group.\n - Otherwise, it is the total number of users minus the count of the group (i.e. the complement).\n\nNote that we skip any groups that would have a count of zero. \n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":683,"value":"$34","pinned":true,"hidden":false,"mode":"ojs"},{"id":765,"value":"md`Here's the output of what we have so far:`","pinned":false,"hidden":false,"mode":"ojs"},{"id":769,"value":"postProcess(anonymize(data.map(processUserRecord)))","pinned":true,"hidden":false,"mode":"ojs"},{"id":761,"value":"md`Finally, we can compute the relevant statistics in a fairly straigtforward manner.`","pinned":false,"hidden":false,"mode":"ojs"},{"id":165,"value":"aggregate = data => {\n const stats = postProcess(data);\n\n const sumRanges = ranges =>\n ranges.reduce(\n (total, { range, count }) => total.add(range.scale(count)),\n Range(0, 0)\n );\n\n const totalSum = sumRanges(stats.sum);\n const totalCount = sumRanges(stats.count);\n\n return {\n min: Range.min(stats.min.map(stat => stat.range)),\n max: Range.max(stats.max.map(stat => stat.range)),\n avg: totalSum.divBy(totalCount),\n median: computeMedian(stats, totalCount)\n };\n}","pinned":true,"hidden":false,"mode":"ojs"},{"id":771,"value":"$35","pinned":false,"hidden":false,"mode":"ojs"},{"id":742,"value":"computeMedian = (stats, totalCount) => {\n const users = reconstructUsers(stats);\n\n const middle = Math.ceil(totalCount.valueOf() / 2);\n let median,\n countSoFar = Range.zero;\n users\n .sort((a, b) => a.avg.valueOf() - b.avg.valueOf())\n .forEach(({ count, avg }) => {\n countSoFar = countSoFar.add(count);\n if (countSoFar.max > middle) {\n median = avg;\n return;\n }\n });\n return median.valueOf();\n}","pinned":true,"hidden":false,"mode":"ojs"},{"id":795,"value":"md`How do we actually figure out the right tuple of (average price, count)? We actually emit 3 important values for each user:\n\n1. The average price ${tex`\\bar p`}\n2. The count of tickets bought ${tex`c`}\n3. The sum of tickets ${tex`\\sum p`}\n\nThese form the relation ${tex`\\sum p = c\\bar p`}, which we can use to associate these. Essentially we need to order these three datasets in a way that all of the numbers are \"explained\".\n\nThis is however a combinatorial problem, and so brute forcing it would be unpheasible for even moderate numbers of users. As such, I use a greedy alogirthm that starts by assigning users with the fewest possibilities and removing those from further users. We pre-order these by the minimal \"error\" in the explanation, so we can minimize backtracking. This seems to result in relatively bearable performance.\n\n\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":784,"value":"$36","pinned":true,"hidden":false,"mode":"ojs"},{"id":811,"value":"md`Some utilities used above:`","pinned":false,"hidden":false,"mode":"ojs"},{"id":815,"value":"pushN = (arr, { range, count }) => {\n for (let i = 0; i < count; i++) {\n arr.push(range);\n }\n return arr;\n}","pinned":true,"hidden":false,"mode":"ojs"},{"id":818,"value":"filterMap = (arr, fn) =>\n arr.reduce((newArr, val, idx) => {\n const result = fn(val, idx);\n if (result) {\n newArr.push(result);\n }\n return newArr;\n }, [])","pinned":true,"hidden":false,"mode":"ojs"},{"id":827,"value":"md`## Results\n\nFinally, this brings us to the end. We've now got all of our statistics: \n\nmin = ${solution.min.toFixed(1)}, \n\nmax = ${solution.max.toFixed(1)}, \n\navg = ${solution.avg.toFixed(1)}\n\nmedian ${solution.median.toFixed(1)}.\n\nWhat exact values you see here depens on the parameters. The defaults differ from the target values by less than 10, however different parameter settings can lead to better or worse accuracy.\n\nCould the accuracy have been better? Yes, but (at least with this approach) it would require pretty excessive resources as each number in the interval would have to be sampled.\n\nAs to trade-offs, I think the worst thing about this solution is that it requires the user to make guesses about the data in terms of the parameters.`","pinned":false,"hidden":false,"mode":"ojs"},{"id":206,"value":"solution = aggregate(anonymize(data.map(processUserRecord)))","pinned":true,"hidden":false,"mode":"ojs"},{"id":838,"value":"md`---\n\n## Docker\n\nYou can download a [little wrapper for this notebook that includes a Dockerfile](https://edinburgh-orthodox.s3.amazonaws.com/aircloack.zip) (for example to run it against another dataset). \n\nHowever, it is not necessary, as this webpage already runs the program and can be easily modified inline.`","pinned":false,"hidden":false,"mode":"ojs"},{"id":61,"value":"md`\n----\n\n## Tests\n\nYou can run the test suite here. Note that since this runs property based tests, this can take a while and the notebook may give an unresponsive warning.\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":64,"value":"$37","pinned":true,"hidden":false,"mode":"ojs"},{"id":125,"value":"md`---\n\n### Plumbing to make this all work`","pinned":false,"hidden":false,"mode":"ojs"},{"id":58,"value":"import { manualTestSuite } from \"@gampleman/jasmine-for-testing\"","pinned":false,"hidden":false,"mode":"ojs"},{"id":97,"value":"fc = require('fast-check@1.17.0/lib/bundle.js')","pinned":false,"hidden":false,"mode":"ojs"},{"id":113,"value":"setupFastCheck = jasmine => {\n jasmine.addMatchers({\n forAll(util, customEqualityTesters) {\n return {\n compare(fn, ...gen) {\n try {\n fc.assert(fc.property(...gen, fn));\n return { pass: true };\n } catch (e) {\n return { pass: false, message: e.message };\n }\n }\n };\n }\n });\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":249,"value":"import { slider } from \"@jashkenas/inputs\"","pinned":false,"hidden":false,"mode":"ojs"},{"id":167,"value":"uniq = (input, getter = id => id) => [\n ...input\n .reduce((map, value) => {\n map.set(JSON.stringify(getter(value)), value);\n return map;\n }, new Map())\n .values()\n]","pinned":false,"hidden":false,"mode":"ojs"}],"files":"$W38","stdlib":"1","titling":"auto"}