Syntactically Valid Code Editing: A Training Methodology for Neural Program Synthesis

Table of Links Abstract and 1. Introduction Abstract and 1. Introduction Background & Related Work Method 3.1 Sampling Small Mutations 3.2 Policy 3.3 Value Network & Search 3.4 Architecture Experiments 4.1 Environments 4.2 Baselines 4.3 Ablations Conclusion, Acknowledgments and Disclosure of Funding, and References Background & Related Work Background & Related Work Background & Related Work Method 3.1 Sampling Small Mutations 3.2 Policy 3.3 Value Network & Search 3.4 Architecture Method Method 3.1 Sampling Small Mutations 3.1 Sampling Small Mutations 3.2 Policy 3.2 Policy 3.3 Value Network & Search 3.3 Value Network & Search 3.4 Architecture 3.4 Architecture Experiments 4.1 Environments 4.2 Baselines 4.3 Ablations Experiments 4.1 Environments 4.1 Environments 4.2 Baselines 4.2 Baselines 4.3 Ablations 4.3 Ablations Conclusion, Acknowledgments and Disclosure of Funding, and References Conclusion, Acknowledgments and Disclosure of Funding, and References Conclusion, Acknowledgments and Disclosure of Funding, and References Appendix Appendix A. Mutation Algorithm A. Mutation Algorithm B. Context-Free Grammars B. Context-Free Grammars C. Sketch Simulation C. Sketch Simulation D. Complexity Filtering D. Complexity Filtering E. Tree Path Algorithm E. Tree Path Algorithm F. Implementation Details F. Implementation Details 3.4 Architecture We add two additional types of tokens: an token, which serves as a start-of-sentence token for the model; and tokens, which allow the model to reference positions within its context. Given a current image, a target image, and a current tokenized program, we train this transformer model to predict the edit position and the replacement text autoregressively. While making predictions, the decoding is constrained under the grammar. We mask out the prediction logits to only include edit positions that represent nodes in the syntax tree, and only produce replacements that are syntactically valid for the selected edit position. We set σsmall = 2, which means the network is only allowed to produce edits with fewer than two primitives. For training data, we sample an infinite stream of random expressions from the CFG. We choose a random number of noise steps, s ∈ [1, 5], to produce a mutated expression. For some percentage of the examples, ρ, we instead sample a completely random new expression as our mutated expression. We trained for 3 days for the environments we tested on a single Nvidia A6000 GPU. Authors: (1) Shreyas Kapur, University of California, Berkeley (srkp@cs.berkeley.edu); (2) Erik Jenner, University of California, Berkeley (jenner@cs.berkeley.edu); (3) Stuart Russell, University of California, Berkeley (russell@cs.berkeley.edu). Authors: Authors (1) Shreyas Kapur, University of California, Berkeley (srkp@cs.berkeley.edu); (2) Erik Jenner, University of California, Berkeley (jenner@cs.berkeley.edu); (3) Stuart Russell, University of California, Berkeley (russell@cs.berkeley.edu). This paper is available on arxiv under CC BY-SA 4.0 DEED license. This paper is available on arxiv under CC BY-SA 4.0 DEED license. available on arxiv available on arxiv

Syntactically Valid Code Editing: A Training Methodology for Neural Program Synthesis

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Thermodynamic Limits Around M-dwarf Stars: Increasing Antenna Size in Limited PAR

From Fixed Labels to Prompts: How Vision-Language Models Are Re-Wiring Object Detection

Rule Engine + LLM Hybrid Architectures for Safer Code Generation

The Topology of Meaning: Towards a "Unified Field Theory" for Artificial Intelligence

Zeno’s Paradox and the Problem of AI Tokenization

Twin‑Tower Neural Architecture for Δ‑Estimation in Differential Financial Models

Thermodynamic Limits Around M-dwarf Stars: Increasing Antenna Size in Limited PAR

From Fixed Labels to Prompts: How Vision-Language Models Are Re-Wiring Object Detection

Rule Engine + LLM Hybrid Architectures for Safer Code Generation

The Topology of Meaning: Towards a "Unified Field Theory" for Artificial Intelligence

Zeno’s Paradox and the Problem of AI Tokenization

Twin‑Tower Neural Architecture for Δ‑Estimation in Differential Financial Models

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps