Chat: Evaluate prompt changes for smart apply

output/chat/claude-3-sonnet/input/chat/question_001/question.yaml

@@ -2,9 +2,14 @@
 # CHAT
 # CHAT_QUESTION implement a dog class
 
-# CHAT_REPLY Based on the provided codebase context, we can implement a `Dog` class that extends the `Animal` interface. Here's an example implementation:
+# CHAT_REPLY Here's an implementation of a `Dog` class that extends the `Animal` interface:
+# CHAT_REPLY 
+# CHAT_REPLY ```typescript:mocked/enhanced_context/animal.ts
+# CHAT_REPLY interface Animal {
+# CHAT_REPLY     name: string;
+# CHAT_REPLY     age: number;
+# CHAT_REPLY }
 # CHAT_REPLY 
-# CHAT_REPLY ```typescript
 # CHAT_REPLY class Dog implements Animal {
 # CHAT_REPLY     name: string;
 # CHAT_REPLY     age: number;
@@ -16,18 +21,18 @@
 # CHAT_REPLY         this.breed = breed;
 # CHAT_REPLY     }
 # CHAT_REPLY 
-# CHAT_REPLY     bark(): void {
-# CHAT_REPLY         console.log(`${this.name} says: Woof!`);
+# CHAT_REPLY     bark() {
+# CHAT_REPLY         console.log("Woof!");
 # CHAT_REPLY     }
 # CHAT_REPLY }
 # CHAT_REPLY ```
 # CHAT_REPLY 
 # CHAT_REPLY In this implementation:
 # CHAT_REPLY 
-# CHAT_REPLY 1. The `Dog` class implements the `Animal` interface, which means it must have the `name` and `age` properties defined in the interface.
-# CHAT_REPLY 2. We add an additional `breed` property to the `Dog` class to store the breed of the dog.
-# CHAT_REPLY 3. The constructor takes `name`, `age`, and `breed` as parameters and initializes the respective properties.
-# CHAT_REPLY 4. We also define a `bark` method that logs a message to the console with the dog's name and a "Woof!" sound.
+# CHAT_REPLY - The `Dog` class implements the `Animal` interface, ensuring that it has the required `name` and `age` properties.
+# CHAT_REPLY - It adds an additional `breed` property to store the breed of the dog.
+# CHAT_REPLY - The constructor takes `name`, `age`, and `breed` as parameters and initializes the respective properties.
+# CHAT_REPLY - A `bark` method is added to the `Dog` class, which logs "Woof!" to the console when called.
 # CHAT_REPLY 
 # CHAT_REPLY You can create instances of the `Dog` class like this:
 # CHAT_REPLY 
@@ -36,10 +41,16 @@
 # CHAT_REPLY console.log(myDog.name); // Output: "Buddy"
 # CHAT_REPLY console.log(myDog.age); // Output: 3
 # CHAT_REPLY console.log(myDog.breed); // Output: "Labrador"
-# CHAT_REPLY myDog.bark(); // Output: "Buddy says: Woof!"
+# CHAT_REPLY myDog.bark(); // Output: "Woof!"
 # CHAT_REPLY ```
-# CHAT_REPLY 
-# CHAT_REPLY This implementation extends the `Animal` interface and adds dog-specific functionality (the `breed` property and the `bark` method) to the `Dog` class.
+
+# CONTEXT_ITEMS [
+# CONTEXT_ITEMS   {
+# CONTEXT_ITEMS     "source": "user",
+# CONTEXT_ITEMS     "file": "/Users/tomross/dev/cody-leaderboard/input/chat/mocked/enhanced_context/animal.ts:undefined-undefined",
+# CONTEXT_ITEMS     "content": "interface Animal {\n    name: string;\n    age: number;\n}\n\n"
+# CONTEXT_ITEMS   }
+# CONTEXT_ITEMS ]
 
  class: generate code
  files:

output/chat/claude-3-sonnet/input/chat/question_002/question.yaml

@@ -2,9 +2,24 @@
 # CHAT
 # CHAT_QUESTION what is squirrel?
 
-# CHAT_REPLY Based on the provided code context, Squirrel is an interface that represents a code intelligence API. It has a single method `symbolInformation` that takes a `file` string as input and returns an object of type `CodeIntelligence`, which contains an array of strings called `symbols`.
+# CHAT_REPLY ```typescript:mocked/enhanced_context/squirrel.ts
+# CHAT_REPLY /**
+# CHAT_REPLY  * Squirrel is a code intelligence API.
+# CHAT_REPLY  */
+# CHAT_REPLY interface Squirrel {
+# CHAT_REPLY     symbolInformation(file: string): CodeIntelligence
+# CHAT_REPLY }
+# CHAT_REPLY ```
 # CHAT_REPLY 
-# CHAT_REPLY The comment above the `Squirrel` interface states that "Squirrel is a code intelligence API", which aligns with the purpose of the `symbolInformation` method to provide information about symbols (e.g., variables, functions, classes) in a given file.
+# CHAT_REPLY Squirrel is a code intelligence API according to the comment in the provided code. It has a `symbolInformation` method that takes a `file` string and returns a `CodeIntelligence` object containing an array of symbols.
+
+# CONTEXT_ITEMS [
+# CONTEXT_ITEMS   {
+# CONTEXT_ITEMS     "source": "user",
+# CONTEXT_ITEMS     "file": "/Users/tomross/dev/cody-leaderboard/input/chat/mocked/enhanced_context/squirrel.ts:undefined-undefined",
+# CONTEXT_ITEMS     "content": "interface CodeIntelligence {\n    symbols: string[]\n}\n\n/**\n * Squirrel is a code intelligence API.\n */\ninterface Squirrel {\n    symbolInformation(file: string): CodeIntelligence\n}\n\n"
+# CONTEXT_ITEMS   }
+# CONTEXT_ITEMS ]
 
  class: codebase onboarding
  files:

output/chat/claude-3-sonnet/input/chat/question_003/question.yaml

@@ -2,11 +2,45 @@
 # CHAT
 # CHAT_QUESTION What code do you have access to?
 
-# CHAT_REPLY Based on the provided README file, I have access to the source code for Zoekt, which is a fast text search engine intended for use with source code. The README provides instructions for downloading, indexing, and searching the codebase using various commands like `zoekt-index`, `zoekt-git-index`, `zoekt-webserver`, and `zoekt`.
+# CHAT_REPLY Based on the provided codebase context from the `README.md` file, I have access to the following code:
 # CHAT_REPLY 
-# CHAT_REPLY The README also mentions that this is a fork of the original Google Zoekt repository (https://github.com/google/zoekt) by Sourcegraph (https://github.com/sourcegraph/zoekt). It outlines some of the differences between this fork and the original, such as the addition of a `zoekt-sourcegraph-indexserver` command, exposing the API via `keegancsmith/rpc`, support for searching across branches/repos, and returning symbol/ctag data in results.
+# CHAT_REPLY ```go:github.com/sourcegraph/zoekt/cmd/zoekt-index/main.go
+# CHAT_REPLY go install github.com/sourcegraph/zoekt/cmd/zoekt-index
+# CHAT_REPLY $GOPATH/bin/zoekt-index .
+# CHAT_REPLY ```
 # CHAT_REPLY 
-# CHAT_REPLY However, the actual source code files are not provided in the context. The README only gives an overview of the project and instructions for using the various components.
+# CHAT_REPLY ```go:github.com/sourcegraph/zoekt/cmd/zoekt-git-index/main.go
+# CHAT_REPLY go install github.com/sourcegraph/zoekt/cmd/zoekt-git-index
+# CHAT_REPLY $GOPATH/bin/zoekt-git-index -branches master,stable-1.4 -prefix origin/ .
+# CHAT_REPLY ```
+# CHAT_REPLY 
+# CHAT_REPLY ```go:github.com/sourcegraph/zoekt/cmd/zoekt-repo-index/main.go
+# CHAT_REPLY go install github.com/sourcegraph/zoekt/cmd/zoekt-{repo-index,mirror-gitiles}
+# CHAT_REPLY zoekt-mirror-gitiles -dest ~/repos/ https://gfiber.googlesource.com
+# CHAT_REPLY zoekt-repo-index /n        -name gfiber /n        -base_url https://gfiber.googlesource.com/ /n        -manifest_repo ~/repos/gfiber.googlesource.com/manifests.git /n        -repo_cache ~/repos /n        -manifest_rev_prefix=refs/heads/ --rev_prefix= /n        master:default_unrestricted.xml
+# CHAT_REPLY ```
+# CHAT_REPLY 
+# CHAT_REPLY ```go:github.com/sourcegraph/zoekt/cmd/zoekt-webserver/main.go
+# CHAT_REPLY go install github.com/sourcegraph/zoekt/cmd/zoekt-webserver
+# CHAT_REPLY $GOPATH/bin/zoekt-webserver -listen :6070
+# CHAT_REPLY ```
+# CHAT_REPLY 
+# CHAT_REPLY ```go:github.com/sourcegraph/zoekt/cmd/zoekt/main.go
+# CHAT_REPLY go install github.com/sourcegraph/zoekt/cmd/zoekt
+# CHAT_REPLY $GOPATH/bin/zoekt 'ngram f:READ'
+# CHAT_REPLY ```
+# CHAT_REPLY 
+# CHAT_REPLY ```go:github.com/sourcegraph/zoekt/cmd/zoekt-indexserver/main.go
+# CHAT_REPLY go install github.com/sourcegraph/zoekt/cmd/zoekt-indexserver
+# CHAT_REPLY ```
+
+# CONTEXT_ITEMS [
+# CONTEXT_ITEMS   {
+# CONTEXT_ITEMS     "source": "user",
+# CONTEXT_ITEMS     "file": "/Users/tomross/dev/cody-leaderboard/input/chat/mocked/codebase_context/README.md:undefined-undefined",
+# CONTEXT_ITEMS     "content": "\n    \"Zoekt, en gij zult spinazie eten\" - Jan Eertink\n\n    (\"seek, and ye shall eat spinach\" - My primary school teacher)\n\nThis is a fast text search engine, intended for use with source\ncode. (Pronunciation: roughly as you would pronounce \"zooked\" in English)\n\n**Note:** This is a [Sourcegraph](https://github.com/sourcegraph/zoekt) fork\nof [github.com/google/zoekt](https://github.com/google/zoekt). It is now the\nmain maintained source of Zoekt.\n\n# INSTRUCTIONS\n\n## Downloading\n\n    go get github.com/sourcegraph/zoekt/\n\n## Indexing\n\n### Directory\n\n    go install github.com/sourcegraph/zoekt/cmd/zoekt-index\n    $GOPATH/bin/zoekt-index .\n\n### Git repository\n\n    go install github.com/sourcegraph/zoekt/cmd/zoekt-git-index\n    $GOPATH/bin/zoekt-git-index -branches master,stable-1.4 -prefix origin/ .\n\n### Repo repositories\n\n    go install github.com/sourcegraph/zoekt/cmd/zoekt-{repo-index,mirror-gitiles}\n    zoekt-mirror-gitiles -dest ~/repos/ https://gfiber.googlesource.com\n    zoekt-repo-index \\\n        -name gfiber \\\n        -base_url https://gfiber.googlesource.com/ \\\n        -manifest_repo ~/repos/gfiber.googlesource.com/manifests.git \\\n        -repo_cache ~/repos \\\n        -manifest_rev_prefix=refs/heads/ --rev_prefix= \\\n        master:default_unrestricted.xml\n\n## Searching\n\n### Web interface\n\n    go install github.com/sourcegraph/zoekt/cmd/zoekt-webserver\n    $GOPATH/bin/zoekt-webserver -listen :6070\n\n### JSON API\n\nYou can retrieve search results as JSON by sending a GET request to zoekt-webserver.\n\n    curl --get \\\n        --url \"http://localhost:6070/search\" \\\n        --data-urlencode \"q=ngram f:READ\" \\\n        --data-urlencode \"num=50\" \\\n        --data-urlencode \"format=json\"\n\nThe response data is a JSON object. You can refer to [web.ApiSearchResult](https://sourcegraph.com/github.com/sourcegraph/zoekt@6b1df4f8a3d7b34f13ba0cafd8e1a9b3fc728cf0/-/blob/web/api.go?L23:6&subtree=true) to learn about the structure of the object.\n\n### CLI\n\n    go install github.com/sourcegraph/zoekt/cmd/zoekt\n    $GOPATH/bin/zoekt 'ngram f:READ'\n\n## Installation\nA more organized installation on a Linux server should use a systemd unit file,\neg.\n\n    [Unit]\n    Description=zoekt webserver\n\n    [Service]\n    ExecStart=/zoekt/bin/zoekt-webserver -index /zoekt/index -listen :443  --ssl_cert /zoekt/etc/cert.pem   --ssl_key /zoekt/etc/key.pem\n    Restart=always\n\n    [Install]\n    WantedBy=default.target\n\n\n# SEARCH SERVICE\n\nZoekt comes with a small service management program:\n\n    go install github.com/sourcegraph/zoekt/cmd/zoekt-indexserver\n\n    cat << EOF > config.json\n    [{\"GithubUser\": \"username\"},\n     {\"GithubOrg\": \"org\"},\n     {\"GitilesURL\": \"https://gerrit.googlesource.com\", \"Name\": \"zoekt\" }\n    ]\n    EOF\n\n    $GOPATH/bin/zoekt-indexserver -mirror_config config.json\n\nThis will mirror all repos under 'github.com/username', 'github.com/org', as\nwell as the 'zoekt' repository. It will index the repositories.\n\nIt takes care of fetching and indexing new data and cleaning up logfiles.\n\nThe webserver can be started from a standard service management framework, such\nas systemd.\n\n\n# SYMBOL SEARCH\n\nIt is recommended to install [Universal\nctags](https://github.com/universal-ctags/ctags) to improve\nranking. See [here](doc/ctags.md) for more information.\n\n\n# ACKNOWLEDGEMENTS\n\nThanks to Han-Wen Nienhuys for creating Zoekt. Thanks to Alexander Neubeck for\ncoming up with this idea, and helping Han-Wen Nienhuys flesh it out.\n\n\n# FORK DETAILS\n\nOriginally this fork contained some changes that do not make sense to upstream\nand or have not yet been upstreamed. However, this is now the defacto source\nfor Zoekt. This section will remain for historical reasons and contains\noutdated information. It can be removed once the dust settles on moving from\ngoogle/zoekt to sourcegraph/zoekt. Differences:\n\n- [zoekt-sourcegraph-indexserver](cmd/zoekt-sourcegraph-indexserver/main.go)\n  is a Sourcegraph specific command which indexes all enabled repositories on\n  Sourcegraph, as well as keeping the indexes up to date.\n- We have exposed the API via\n  [keegancsmith/rpc](https://github.com/keegancsmith/rpc) (a fork of `net/rpc`\n  which supports cancellation).\n- Query primitive `BranchesRepos` to efficiently specify a set of repositories to\n  search.\n- Allow empty shard directories on startup. Needed when starting a fresh\n  instance which hasn't indexed anything yet.\n- We can return symbol/ctag data in results. Additionally we can run symbol regex queries.\n- We search shards in order of repo name and ignore shard ranking.\n- Other minor changes.\n\nAssuming you have the gerrit upstream configured, a useful way to see what we\nchanged is:\n\n``` shellsession\n$ git diff gerrit/master -- ':(exclude)vendor/' ':(exclude)Gopkg*'\n```\n\n# DISCLAIMER\n\nThis is not an official Google product\n"
+# CONTEXT_ITEMS   }
+# CONTEXT_ITEMS ]
 
  class: cody usage
  files: