Waypoint

The problem

Manual web work is expensive and error-prone, and one-off scripts are brittle. Teams need consistent, shareable browser workflows that survive site changes — not a pile of scripts only one person understands.

What it does

Built with Deeksha Ravi and Nyan Lin Htet as Team Next Horizon. Describe a task in plain language, and Waypoint plans the steps, runs a real browser live with streaming screenshots so you can watch it work, and packages the result into a link anyone on the team can rerun with zero code.

• Live local browser runs with streaming screenshots as the automation executes.
• Shareable automation links, so a workflow built once can be handed to anyone.
• A saved automation library in the browser for reuse.
• Prompt-based step estimation before a run starts.
• A tool-driven agent loop that keeps browser actions controlled rather than free-form.

How it's built

Gemini is the decision engine — it interprets the plain-language task, plans the browser actions, and decides the optimal next step at every turn. Playwright is the execution layer underneath, handling the actual clicking, typing, and navigation Gemini directs.

• FastAPI backend, React/Vite frontend.
• Playwright for browser automation.
• Google Gemini API for task planning and decision-making.
• Deployed live on Render, with session logging and screenshot capture.

Ethics and human-in-the-loop

• When Waypoint hits a CAPTCHA or similar security check, it pauses and waits for a human rather than trying to bypass it.
• Users can constrain automations with plain-language rules — "only visit these domains," "stop if you see this message" — so a run stays inside intended boundaries.

Placed 3rd at the Frontiers Hackathon, hosted with Google DeepMind.