## Summary

I demonstrate an imaginary linter with Codex. This predicts an error message for a line of awk code. However, it also infers when the code is correct, so it’s useful.

The accuracy is not really good enough to make a reliable imaginary linter, but should work better with more popular languages such as Python.

In any case, I will build the tooling around prompts to perform linting.

## Demo

### Correctly identifies syntax error 7 times out of 20

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20  6: } 5: ^ syntax error 6: } 5: ^ syntax error 5: ^ syntax error 5: ^ 6: } 5: ^ syntax error 6: } 6: } 6: } 6: } 6: } 6: } 6: } 5: ^ syntax error 6: } 5: ^ syntax error 6: } 5: ^ syntax error

### Correctly identifies correct code 19 times out of 20

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20  6: } 6: } 6: } 6: } 6: } 6: } 6: } 6: } 6: } 6: } 6: } 6: } 6: } 6: } 6: } 6: } 6: } 6: } 6: } 5: ^ syntax error

## Prompt

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33  task: "Imagine an awk linter" doc: "Given a line of awk code, check it for errors" issues: - This doesn't work very well at all. prompt-version: 1 prompt: |+ awk -f syntax-test.awk awk: syntax-test.awk:1: x = f( awk: syntax-test.awk:1: ^ unexpected newline or end of string awk: syntax-test.awk:2: for (i=1; i awk: syntax-test.awk: engine: "OpenAI Codex" temperature: 0.0 max-tokens: 60 n-completions: 10 no-uniq-results: on top-p: 1.0 cache: on stop-sequences: - "\n" vars: - "line of code" examples: - "close cmd, \"to\"" preprocessors: - "cat" postprocessor: "grep -v -P \"^6\" | sed \"s/^5: //\"" filter: yes completion: off insertion: off