Code Alpaca

Contents

Code Alpaca#

Note

Code Alpaca is fully based on Stanford Alpaca ,and only changes the data used for training.

  • Modified prompt to focus on code generation/editing/optimization tasks instead of general tasks.

  • Modified seed tasks to only be related to code generation.

This produced an instruction-following dataset with 20K examples obtained at a much lower cost.

You are asked to come up with a set of 20 diverse code generation task instructions. These task instructions will be given to a GPT model and we will evaluate the GPT model for completing the instructions.

Here are the requirements:
1. Try not to repeat the verb for each instruction to maximize diversity.
2. The language used for the instruction also should be diverse. For example, you should combine questions with imperative instrucitons.
3. The type of instructions should be diverse. The list should include diverse types of programming tasks like open-ended generation, classification, editing, optimization etc.
2. A GPT language model should be able to complete the instruction. For example, do not ask the assistant to create any visual or audio output. For another example, do not ask the assistant to wake you up at 5pm or set a reminder because it cannot perform any action.
3. The instructions should be in English.
4. The instructions should at least 1 to 2 sentences long. Either an imperative sentence or a question is permitted.
5. You should generate an appropriate input to the instruction. The input field should contain a specific example provided for the instruction. It should involve realistic data and should not contain simple placeholders. The input should provide substantial content to make the instruction challenging but should ideally not exceed 100 words.
6. Not all instructions require input. For example, when a instruction asks about some general information, "write a python program to load a file.", it is not necssary to provide a specific context. In this case, we simply put "<noinput>" in the input field.
7. The output should be an appropriate response to the instruction and the input.
8. All tasks should be coding or programming related.

List of 20 tasks:

Stanford Alpaca#

Stanford Alpaca is fine-tuned from a 7B LLaMA model on 52K instruction-following data generated by the techniques in the Self-Instruct, with some modifications.

  • We used text-davinci-003 to generate the instruction data instead of davinci.

  • We wrote a new prompt that explicitly gave the requirement of instruction generation to text-davinci-003.

  • We adopted much more aggressive batch decoding, i.e., generating 20 instructions at once, which significantly reduced the cost of data generation.

  • We simplified the data generation pipeline by discarding the difference between classification and non-classification instructions.

  • We only generated a single instance for each instruction.

Contents