## Introducing ESP (A language server for everything)

ESP is a Language server powered by Pen. It’s a portmanteau backronym of Everything language Server Protocol.

Using any language model, one can obtain documentation and editing facilities for any computing context.

“One LSP server to rule all the computing contexts! And in the emacs bind them.” - Sauron

I set up an LSP server for OpenAI’s GPT-3/Codex or any LM via your favourite NLP service provider.

First LSP server for LMs in the world, as far as I know.

It gives documentation and refactoring to any language including world languages and fictional ones. As an LSP server, it works for VSCode too.

The LSP server can create hover documentation, code actions and linting services for any programming language, world language, fictional language or computing context.

## See it in action

I provide some introspection into the LSP protocol stream.

### Caching requests

Documentation is instantaneous with caching enabled.

Obviously, this means that a centralised cache of truthful documentation is now a necessity.

But how do we have consensus on what is a high quality suggestion/generation from a LM?

• enter blockchain or Ministry of Truth (you decide)

## How does it work?

Pen.el utilises the efm-langserver along with its shell-interop.

Essentially, you design documentation and refactoring functions (prompt-functions) like so.

https://semiosis.github.io/posts/pen-el-host-interop-and-client-server/

### efm-langserver

Then you configure EFM langserver like so:

https://github.com/mattn/efm-langserver

### EFM Config

 1 2 3  glossary1: &glossary1 hover-command: '"penf" "pf-define-word-for-glossary/1"' hover-stdin: true

### emacs lisp

The LSP client communicates with the LSP server.

## Demo

Hovering over “The Anatomy of a Monad”.

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15  Notebook The The definite article used before a noun. Anatomy The branch of biology concerned with the study of the structure of organisms and their parts. of Indicating possession. a Used to form the plural of most nouns. Monad The smallest unit of matter that exists.

## Implementing completion

### Ensure pen can receive the POSITION

Then remove the parts of the text which are not required, and forward the rest to the prompt function.

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23  stdin_exists() { ! [ -t 0 ] && ! test "$(readlink /proc/$$/fd/0)" = /dev/null } if stdin_exists; then # 0/dev/null # cmd "$@" | tv &>/dev/null input_fp="$(cat | tf txt)" if test -n "$POSITION"; then line="$(echo "$POSITION" | cut -d : -f 1)" col="$(echo "$POSITION" | cut -d : -f 2)" this_line="$(( line + 1 ))" next_line="$(( line + 2 ))" sed -i "$next_line,\$d" "$input_fp" sed -i "${this_line}s/^\$$.\\{col\\}\$$.*\$/\\1/" "$input_fp" fi exec < <(cat "$input_fp") fi  1 2 3 4 5  tools: pen-world-language-completion: &pen-world-language-completion # completion-command: 'ci penf pf-generic-completion-50-tokens/1' completion-command: 'POSITION=${POSITION} penf -u pf-generic-completion-50-tokens/1' completion-stdin: true

### Study the racket language server

• see how output is formed
  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25  #!/bin/bash export TTY # raco pkg update --batch racket-langserver if myrc-test ask_view_racket_lsp; then tm yn "view stdin?" && exec < <(tmicat) tm yn "view stdout?" && exec 1> >(tmicat) elif myrc-test ask_log_racket_lsp; then tm yn "log stdin?" && { tf_in="$(ux tf in || echo /dev/null)" exec < <(tee "$tf_in") 0 >(tee "$tf_out") 0 >(tmi) fi "racket" "--lib" "racket-langserver" "$@"

examining racket langserver output

This json was sent by the client to the server:

## pen-lsp-complete

### emacs lisp

Because efm-langserver expects one completion per line from the shell script is uses (rather than json), I had to make a workaround.

When I provide efm-langserver with completion candidates, I one-linerize them by replacing “\n” with <pen-newline>.

Since efm-langserver is not providing the appropriate textEdit to unonlinerize, I have to do it myself in company-lsp.

Unfortunately, company-lsp uses lexical scope, so I had to copy the entire file across and rename it to add the functionality.

https://github.com/semiosis/pen.el/blob/master/src/pen-company-lsp.el#L346

## Final product

### Demo

This is a demo of both documentation and completion using the Pen.el language server.

### Built into the docker image

EFM Langserver is also built into the Pen.el docker image.