This title slide presents the talk titled "Developing a Minor Embedding Algorithm for Oscillator-Based Ising Machines Using Reinforcement Learning." The subtitle lists the presenter Shishir Siwakoti, along with affiliation and date.

Developing a Minor Embedding Algorithm for Oscillator-Based Ising Machines Using Reinforcement Learning

Shishir Siwakoti [Affiliation] [Date]

Source: Dark tech background with abstract oscillator visuals

Ising machines revolutionize combinatorial optimization, but minor embedding on oscillator networks remains challenging. Current methods lack efficiency and scalability, highlighting the need for automated algorithms to enable practical deployment.

Motivation

• Ising machines revolutionize combinatorial optimization problems
• Minor embedding on oscillator networks is challenging
• Current methods lack efficiency and scalability
• Need automated algorithms for practical deployment

Source: Developing a Minor Embedding Algorithm for Oscillator-Based Ising Machines Using Reinforcement Learning by Shishir Siwakoti

The slide titled "Objectives" outlines the development of an RL-based minor embedding algorithm optimized for oscillator-based hardware constraints and tailored to Ising machine dynamics. It aims to achieve fast, scalable embeddings that enable efficient solving of optimization problems.

Objectives

• Develop RL-based minor embedding algorithm
• Optimize for oscillator-based hardware constraints
• Achieve fast and scalable embeddings
• Enable efficient solving of optimization problems
• Tailor policy to Ising machine dynamics

The slide defines the problem of minor embedding in QUBO formulations, which map to Ising models for oscillator hardware by assigning logical qubits to physical oscillators via chains, with scalability and quality as key challenges. It highlights oscillator networks' limited connectivity (e.g., ring or grid topologies), necessitating embeddings that respect these constraints and minimize chain lengths to curb errors from injection locking and noise.

Problem Definition

Source: Developing a Minor Embedding Algorithm for Oscillator-Based Ising Machines Using Reinforcement Learning by Shishir Siwakoti

This slide illustrates a hardware model where coupled oscillators represent Ising spins and couplings, with oscillator phases encoding binary spin states. Readout layers extract states from phase differences, while injection locking synchronizes oscillators through interactions.

Hardware Model

!Image

• Coupled oscillators represent Ising spins and couplings.
• Phases of oscillators encode binary spin states.
• Readout layers extract states from phase differences.
• Injection locking synchronizes oscillators via interactions.

Source: Coherent Ising machine

This slide defines RL components for minor embedding: states as partial logical qubit embeddings, actions as chain extensions (e.g., adding auxiliary oscillators), and rewards as embedding quality scores penalized by chain length. These elements form a complete MDP with deterministic transitions to train policies balancing solution accuracy and hardware efficiency.

RL Formulation

{ "headers": [ "Component", "Definition", "Purpose" ], "rows": [ [ "Define States", "Partial embeddings of logical qubits", "Represent current chain configuration and embedding progress" ], [ "Define Actions", "Chain extensions (e.g., add auxiliary oscillators)", "Explore and expand minor embedding space" ], [ "Define Rewards", "Embedding quality score - penalty for chain length", "Balance solution accuracy and hardware efficiency" ], [ "Formulate MDP", "States + Actions + Rewards + Transitions (deterministic extensions)", "Complete RL environment for policy training" ] ] }

Source: Thesis: Developing a Minor Embedding Algorithm for Oscillator-Based Ising Machines Using Reinforcement Learning by Shishir Siwakoti

The "Policy Architecture" slide features a grid of five key components: Actor-Critic Network for stable RL, Graph Neural Nets for qubit connectivity encoding, Policy Head for action probabilities, Value Head for return predictions, and Attention Mechanisms for focused decision-making. It highlights how these elements combine actor policies, value estimation, graph processing, and attention to optimize minor embedding decisions.

Policy Architecture

{ "features": [ { "icon": "🤖", "heading": "Actor-Critic Network", "description": "Combines actor for policy and critic for value estimation in stable RL." }, { "icon": "🌐", "heading": "Graph Neural Nets", "description": "Encodes qubit connectivity states using message-passing and aggregation." }, { "icon": "🎯", "heading": "Policy Head", "description": "Outputs action probabilities for minor embedding decisions." }, { "icon": "📈", "heading": "Value Head", "description": "Predicts expected returns to compute policy gradients accurately." }, { "icon": "🔍", "heading": "Attention Mechanisms", "description": "Focuses on relevant graph nodes for enhanced decision-making." } ] }

The timeline details training starting with pre-training on small graphs in 2022 for foundational embedding skills, followed by fine-tuning on medium-sized graphs in Q1 2023 for complex minor embedding tasks. Benchmark evaluation occurred in Q2 2023, assessing performance on Chimera and Upham graphs using standard Ising machine metrics.

Training and Evaluation

2022: Pre-training on Small Graphs Initial RL policy training on small graph instances to build foundational embedding skills. 2023 Q1: Fine-tuning the Policy Refined model on medium-sized graphs, optimizing for complex minor embedding tasks. 2023 Q2: Benchmark Evaluation Assessed performance on Chimera and Upham graphs using standard Ising machine metrics.

The Results slide showcases a 95% embedding success rate on test instances and a 20% average chain length reduction compared to initial embeddings. It also reports 3x faster performance than baselines, with reduced computation time.

Results

• 95%: Embedding Success Rate

Achieved on test instances

• 20%: Avg Chain Length Reduction

Compared to initial embeddings

• 3x: Faster than Baselines

Reduced computation time

This table compares four methods—RL Embedding (Ours), Classical Heuristics, D-Wave, and Other ML Methods—on success rate, average time, and scalability. RL Embedding achieves the top 95% success rate with high scalability (200+), outperforming others despite a moderate 12s average time.

Comparisons

{ "headers": [ "Method", "Success Rate (%)", "Avg. Time (s)", "Scalability" ], "rows": [ [ "RL Embedding (Ours)", "95", "12", "High (200+)" ], [ "Classical Heuristics", "78", "8", "Medium (100)" ], [ "D-Wave", "88", "2", "High (5000)" ], [ "Other ML Methods", "90", "25", "Medium (150)" ] ] }

RL achieves superior minor embeddings, remains robust to hardware constraints, and advances Ising machine usability. The slide concludes that RL revolutionizes Ising machine embeddings and urges applying RL to advance quantum-inspired hardware.

Conclusions

• RL achieves superior minor embeddings

• Robust to hardware constraints
• Advances Ising machine usability

Closing: RL revolutionizes Ising machine embeddings.

Call-to-Action: Apply RL to advance quantum-inspired hardware!

Key Achievements & Impact

Source: Developing a Minor Embedding Algorithm for Oscillator-Based Ising Machines Using Reinforcement Learning by Shishir Siwakoti

The "Future Work" slide outlines plans to scale the embedding algorithm for larger Ising problems and develop hybrid classical-quantum optimization approaches. It also proposes conducting experiments on real oscillator hardware and incorporating multi-objective reinforcement learning.

Future Work

• Scale embedding algorithm to larger Ising problems
• Develop hybrid classical-quantum optimization approaches
• Conduct experiments on real oscillator hardware
• Incorporate multi-objective reinforcement learning