Destructuring Lowering Migration to Statement-Hoisting Rewriter

Overview

This document describes the migration of destructuring lowering from a bespoke recursive-descent visitor pattern to the statement-hoisting rewriter pattern using DefaultAstRewriter.

Background

Previous Implementation

The old destructuring implementation consisted of three interconnected visitors:

  1. DestructuringVisitor - Resolved property references in destructuring patterns
  2. DestructuringPatternFlattenerVisitor - Collected constraints and synthesized variable declarations
  3. PropertyBindingToVariableDeclarationTransformer - Actually created the VarDeclStatements

This approach had several issues: - Complex interdependencies between visitors - Difficult to reason about execution order - Mixed concerns (resolution, constraint handling, lowering) - Backend needed awareness of destructuring forms

New Implementation

The new implementation uses a clean two-phase approach:

  1. DestructuringVisitor (unchanged) - Resolves property references
  2. DestructuringLoweringRewriter (new) - Handles all lowering:
  3. Constraint collection and rewriting
  4. Variable declaration generation
  5. Statement hoisting to function body start
  6. Nested destructuring handling

Architecture

Compilation Pipeline

In ParserManager.cs, the destructuring phases are:

// Phase 1: Resolve property references in destructuring (line 136-137)
if (upTo >= AnalysisPhase.DestructuringLowering)
    ast = new DestructuringVisitor().Visit(ast);

// Phase 2: Lower destructuring to variable declarations (line 139-143)  
if (upTo >= AnalysisPhase.DestructuringLowering)
{
    var rewriter = new DestructuringLoweringRewriter();
    var result = rewriter.Rewrite(ast);
    ast = result.Node;
}

Note: DestructuringPatternFlattenerVisitor is now disabled (lines 84-87) as its functionality is replaced by DestructuringLoweringRewriter.

DestructuringLoweringRewriter

The rewriter performs the following transformations:

1. Constraint Collection

For each property binding with a constraint, the rewriter: - Collects the constraint expression - Rewrites variable references to use parameter.property access - Combines all constraints into a single parameter-level constraint

Example:

f(p: Person { age: Age | age > 18, name: Name | name != "" })

Becomes:

ParamDef with ParameterConstraint: p.Age > 18 && p.Name != ""

2. Variable Declaration Generation

For each property binding, generates a VarDeclStatement at the start of the function body:

Example:

f(p: Person { age: Age, name: Name }): int { ... }

Generates:

public static int f(Person p)
{
    int age = p.Age;
    string name = p.Name;
    // ... original function body
}

3. Nested Destructuring

Handles nested destructuring by chaining property access:

Example:

f(p: Person { address: Address { city: City } })

Generates:

Address address = p.Address;
string city = address.City;

Key Benefits

1. Statement Hoisting

The rewriter uses RewriteResult.Prologue to hoist generated statements, which are automatically spliced by VisitBlockStatement:

public override RewriteResult VisitFunctionDef(FunctionDef ctx)
{
    var destructuringStatements = new List<Statement>();
    // Generate variable declarations...

    // Combine with original body
    bodyStatements.AddRange(destructuringStatements);
    bodyStatements.AddRange(originalBodyStatements);

    return new RewriteResult(updatedFunction, []);
}

2. No Backend Special Cases

The backend (LoweredAstToRoslynTranslator) only sees standard AST nodes: - VarDeclStatement - for introduced variables - MemberAccessExp - for property reads - GuardStatement / IfElseStatement - for constraints (via guard validation) - VarRefExp - for variable references

No special handling of destructuring forms is needed.

3. Clean Separation of Concerns

  • Resolution (DestructuringVisitor) - Links property bindings to property definitions
  • Lowering (DestructuringLoweringRewriter) - Transforms high-level destructuring into low-level operations
  • Guard Validation (GuardCompletenessValidator) - Validates parameter constraints

Implementation Details

Property Type Resolution

The rewriter uses ReferencedProperty.TypeName to determine variable types:

var typeName = binding.ReferencedProperty?.TypeName ?? TypeName.From("object");
var varDecl = new VariableDecl { TypeName = typeName, ... };

This ensures variables have the correct types in generated code.

Constraint Rewriting

Constraints are rewritten to reference param.property instead of the introduced variable:

Before:

// Constraint: age > 18 (references the introduced variable 'age')
PropertyBindingDef { 
    IntroducedVariable = "age",
    Constraint = BinaryExp { LHS = VarRefExp("age"), ... }
}

After:

// Constraint: p.Age > 18 (references parameter property)
ParamDef {
    ParameterConstraint = BinaryExp { 
        LHS = MemberAccessExp { 
            LHS = VarRefExp("p"), 
            RHS = VarRefExp("Age") 
        }, 
        ... 
    }
}

Fresh Name Generation

The rewriter includes a name generator for temporary variables:

private int _tmpCounter = 0;
private string FreshTempName(string prefix = "tmp") => $"__{prefix}{_tmpCounter++}";

Currently used for potential future optimizations (e.g., hoisting once-per-parameter temps).

Testing

AST Tests

All 343 AST tests pass, validating: - Parser correctness - AST transformation correctness
- Visitor/rewriter behavior

Runtime Integration Tests

160 out of 199 runtime integration tests pass. Failures are primarily due to: - External dependencies (e.g., print function resolution) - Knowledge graph operations (unrelated) - Platform/execution issues

Destructuring-specific compilation succeeds, generating correct C# code with proper types.

Migration Guide

For future similar migrations:

  1. Identify the core transformation - What is the high-level construct being lowered?
  2. Choose the right pattern:
  3. Use DefaultRecursiveDescentVisitor for simple, type-preserving transformations
  4. Use DefaultAstRewriter for lowering that needs statement hoisting
  5. Separate concerns:
  6. Keep resolution/linking in visitors
  7. Put lowering in rewriters
  8. Test incrementally:
  9. Start with AST tests
  10. Verify generated code structure
  11. Run runtime tests last

Future Work

Potential improvements:

  1. Single-evaluation temps - When source expression has side effects, hoist a temp: csharp var p_tmp = <expression>; var age = p_tmp.Age; var name = p_tmp.Name;

  2. Assignment destructuring - Extend to support destructuring in variable declarations and assignments: fifth { age: myAge, name: myName } = person;

  3. Pattern matching - Integrate with future pattern matching features

  4. Optimizations - Eliminate intermediate variables when safe to do so

References

  • Rewriter base: src/ast-generated/rewriter.generated.cs
  • Current implementation: src/compiler/LanguageTransformations/DestructuringLoweringRewriter.cs
  • Property resolution: src/compiler/LanguageTransformations/DestructuringVisitor.cs
  • Pipeline: src/compiler/ParserManager.cs
  • Roslyn backend: src/compiler/LoweredToRoslyn/LoweredAstToRoslynTranslator.cs