31:T4c5,md ` ||| |---|---| |**TP**
true positive |**FP**
false positive | |**TN**
true negative |**FN**
false negative | |**total**
total number of samples| = TP + FP + TN + TP| |**accuracy**
how the model performs on predictions |${tex `= \Large \frac{\text{TP} + \text{TN}}{\text{total}}`}| |**error**
= 1 - accuracy| = ${tex ` \Large \frac{\text{FP} + \text{FN}}{\text{total}}`}| |**precision**
how precise/accurate the model is to predict positive values|${tex `= \Large \frac{\text{TP}}{\text{TP} + \text{FP}}`}| |**recall, sensitivity**
measures the proportion of actual positives that are correctly
predicted as such|${tex `= \Large \frac{\text{TP} }{\text{TP} + \text{FN}}`}| |**specificity**
measures the proportion of actual negatives that are correctly
predicted as such|${tex `= \Large \frac{\text{TN} }{\text{TN} + \text{FP}}`}| |**F1 score**
is the harmonic mean of precision and sensitivity|= ${tex ` 2 \cdot \Large\frac{\text{precision} \;\cdot\; \text{recall}}{(\text{precision} + \text{recall})}`} | `32:[] 2f:{"title":"Structuring Machine Learning Projects","cells":[{"id":0,"value":"md `# Structuring Machine Learning Projects\nPersonal notes used during a course in \"Deep Learning\"\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":124,"value":"md `\n## Train / Dev / Test Distributions\n\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":269,"value":"md `

`","pinned":false,"hidden":false,"mode":"ojs"},{"id":304,"value":"md `#### train-dev set\nSame distribution as training set, but not used for training.\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":402,"value":"md `

\n## Error Analysis\n\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":397,"value":"md `

`","pinned":false,"hidden":false,"mode":"ojs"},{"id":407,"value":"md ` \n### Analysis\n|#||Error %| |Definition / Diagnosis|\n|---|---|---|---|---|\n|0|${bullet(\"white\")}|Bayes Optimal Error| | *lowest possible error rate*|\n|0|${bullet(\"white\")}|Human Performance
Level %| |*proxy for Bayes Error in most cases of computer vision*|\n|1|${bullet(\"#aeb6bfff\")}|Training Set Error %|Δ(#1 - #0)| **Avoidable Bias**|\n|2|${bullet(\"#aeb6bfff\")}|Training-Dev Set
Error %| Δ(#2 - #1)| **Variance Problem** (overfitting)|\n|3|${bullet(\"#a2d9ceff\")} |Dev Set Error %|Δ(#3 - #1)
Δ(#3 - #2)| **Variance Problem** (overfitting) or/and **Data Mismatch**
**Data Mismatch** |\n|4|${bullet(\"#a2d9ceff\")}|Test Set Error %| Δ(#4 - #3)| **Overfitting the Dev Set**|\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":513,"value":"md ` \n### Improve Model Performance\n|Diagnosis|Actions|\n|---|---|\n|**Avoidable Bias**| → train bigger model
→ train longer
→ better optimization algorythm (**Momentum**, **RMS Prop**, **Adam**, ...)
→ NN Architecture (activation functions, **CNN**, **RNN**, ...)|\n|**Variance**| → get more data
→ data augmentation
→ Regularization (**L2**, **Dropout**, ...)
→ NN Architecture (activation functions, **CNN**, **RNN**, ...)|\n|**Data Mismatch**| → carry out manual error analysis
→ make the training data more similar to your dev set
→ cleaning up incorrectly labeled data|\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":307,"value":"md `

\n## Metrics\n\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":312,"value":"$31","pinned":false,"hidden":false,"mode":"ojs"},{"id":12,"value":"md `

`","pinned":false,"hidden":false,"mode":"ojs"},{"id":475,"value":"bullet = (color) => `•`","pinned":false,"hidden":false,"mode":"ojs"},{"id":275,"value":"function swatch(color, title) {\n return html`

${title}

`;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":22,"value":"import {titleColor} from \"@ig2gi/supervised-learning-deep-learning-part1\"","pinned":false,"hidden":false,"mode":"ojs"},{"id":19,"value":"import {formulaBgColor} from \"@ig2gi/supervised-learning-deep-learning-part1\"","pinned":false,"hidden":false,"mode":"ojs"},{"id":15,"value":"import {formulaTxtColor} from \"@ig2gi/supervised-learning-deep-learning-part1\"","pinned":false,"hidden":false,"mode":"ojs"},{"id":9,"value":"d3 = require(\"d3@5\")","pinned":false,"hidden":false,"mode":"ojs"}],"files":"$W32","stdlib":"1","titling":"auto"}