Agent runtime survey
Each agent answers the same standard questionnaire inside its own runtime, mostly by running commands. The main table keeps only the indicators that matter for choosing; full answers are under Test evidence below.
Data updated: 2026-10-15 09:43RUNTIME
At a glance
Colour: green = good for long-running agents, amber = conditional, red = unfavourable. Task–runtime consistency = whether the agent’s self-report matches our own tests.
| Indicator | Meta Muse2026-10-15 | Grok Bot2026-10-15 | LightVela (Tencent Lighthouse)2026-10-15 |
|---|---|---|---|
| Runtime typeQ 1 | VM (sandbox) | VPS (cloud computer) | VPS (cloud computer) |
| Machine persistsQ 15, 30 | No, VM replaced often | Yes, long-term per user (restarts) | Yes, one long-lived VM (up 2+ weeks) |
| Workspace persistenceQ 13 | Partly, home directory kept | Kept | Kept |
| Software persistenceQ 14 | Partly, only under ~/workspace | Kept | Kept |
| GUI / browserQ 10, 11 | No desktop · browser control | Desktop · browser | Desktop on demand · browser control |
| IP stabilityQ 17, 18, 19, 20 | Low: datacenter, dynamic, geo drift | Medium: datacenter, changes within one range | High: Tencent Cloud datacenter IP, unchanged 2+ weeks |
| Background workQ 21, 22 | While the VM lives · platform scheduler | Until restart · platform scheduler | Yes, 2+ weeks uninterrupted · system cron |
| Lifecycle awarenessQ 16 | Can detect VM replacement | Partly, via boot time | Partly, via uptime and files |
| Task–runtime consistency | Consistent: VM and dynamic egress match our tests | Consistent: VPS, kept files and dynamic egress match our tests | Partly: long-lived VM and stable IP match our tests; the agent calls itself a sandbox |
| User takeoverQ 26 | No | Desktop takeover in the app | Limited: SSH/VNC run but need a tunnel |
| Fit for long-running agents | Partly | Yes | Yes |
EVIDENCE
Test evidence
Open a product to see all 30 answers. Source labels are the basis the agent gave (command output, platform docs, past observation, statement, inference, unknown); items that match our own tests are marked.
Meta MuseAll 30 answers · questionnaire V1.0 · 2026-10-15
| # | Question | Answer | Source |
|---|---|---|---|
| Base environment | |||
| 1 | Environment type | VM (sandbox)The platform tags the task context vm=standard, a managed VM sandbox. | Agent: platform docs matches our test |
| 2 | Own operating system | YesOwn Linux userspace; can run any command and long-lived processes. | Agent: command output |
| 3 | Operating system | Ubuntu 24.04.5 LTS (x86_64) | Agent: command output |
| 4 | CPU | 2 vCPU, AMD EPYC 9D25 | Agent: command output |
| 5 | Memory | 7.7 GiB | Agent: command output |
| 6 | Disk | 100 GB home partition (about 98 GB free); /tmp is a 512 MB tmpfs | Agent: command output |
| Permissions & tools | |||
| 7 | System privileges (root / administrator) | Full (root) | Agent: command output |
| 8 | Install software yourself (apt / pip / npm) | YesAnything under ~/workspace persists; system-wide installs disappear when the VM is replaced. | Agent: command output |
| 9 | Run shell commands directly | Yes | Agent: command output |
| 10 | Graphical desktop | NoHeadless; Xvfb is only for browser automation. | Agent: command output |
| 11 | Launch and control a browser | YesDrives Chromium through platform browser tasks; a new browser process per task. | Agent: past observation |
| State persistence | |||
| 12 | Browser logins kept across sessions | PartlyKept while the same VM lives; lost when it is replaced. | Agent: past observation |
| 13 | Files kept across sessions | PartlyThe home directory survives VM replacement; /tmp and system directories do not. | Agent: past observation |
| 14 | Installed software kept across sessions | PartlyTools under ~/workspace persist; system-level installs do not. | Agent: past observation |
| 15 | VM may be deleted, rebuilt or replaced by the platform | YesPlatform deploys are frequent (often hourly), so the VM can be replaced at any time. | Agent: platform docs |
| 16 | Can it tell when the VM is replaced | YesMissing non-home files, a changed machine-id, or a jump in egress IP. | Agent: past observation |
| Network & IP | |||
| 17 | Public egress IP static or dynamic | DynamicBrowser egress rotates within a proxy pool. | Agent: past observation matches our test |
| 18 | Egress IP same across sessions / restarts | No | Agent: past observation matches our test |
| 19 | Egress IP type | DatacenterSeen as Fastly AS54113 and Cloudflare AS13335; shell and browser use different egress, and the browser one is what sites see. | Agent: past observation |
| 20 | Egress country / region may change | May changeThe same IP geolocates differently across databases (Virginia / New York / San Francisco / Montreal). | Agent: past observation |
| Long-running capability | |||
| 21 | Background processes survive the task | SometimesSurvives while the VM lives; stops when the VM is replaced. | Agent: platform docs |
| 22 | Scheduled jobs | Via platform schedulerPlatform-managed cron / hooks, not the system crontab. | Agent: platform docs |
| 23 | Longest run time | UnknownA single task is capped at 86,400 seconds; how long a VM lives is unknown. | Agent: unknown |
| 24 | Paused or reclaimed when idle | YesVMs are replaced on platform deploys, regardless of idleness. | Agent: platform docs |
| 25 | Snapshot / resume | NoNo whole-machine snapshot; home files persist and the platform resumes scheduled tasks. | Agent: platform docs |
| Access & services | |||
| 26 | User can enter via SSH / RDP / VNC | NoNo inbound ports are open. | Agent: platform docs |
| 27 | Outbound internet access | YesSome sites block datacenter IPs. | Agent: command output |
| 28 | Listen on a port reachable from the internet | NoNo public IP; only via a reverse tunnel. | Agent: platform docs |
| 29 | Credentials kept long-term across tasks | YesCredentials in the Secure Vault persist across tasks and VM replacements. | Agent: platform docs |
| 30 | Who the environment belongs to | Long-term per userThe home directory is long-term and per user, but the compute instance underneath gets replaced. | Agent: platform docs |
The agent’s own summary
- Environment type: VM (sandbox)
- Persistence: Medium
- IP stability: Low
- Self-awareness: High
- Fit for long-running agent work: Partly
Home directory and credentials survive VM replacement and platform schedules run long-term, which suits orchestration; but the compute instance is replaced at any time and the egress is a dynamic, geo-drifting datacenter IP, so it does not suit fixed-IP, long-lived process or inbound-service work.
Grok BotAll 30 answers · questionnaire V1.0 · 2026-10-15
| # | Question | Answer | Source |
|---|---|---|---|
| Base environment | |||
| 1 | Environment type | VPS (cloud computer)A long-lived Linux VM. | Agent: command output matches our test |
| 2 | Own operating system | Yes | Agent: command output |
| 3 | Operating system | Debian 13 (trixie) | Agent: command output |
| 4 | CPU | 8 vCPU, Intel Xeon | Agent: command output |
| 5 | Memory | 15 GiB (about 16 GB) | Agent: command output |
| 6 | Disk | 125 GB (120 GB free) | Agent: command output |
| Permissions & tools | |||
| 7 | System privileges (root / administrator) | Full (regular user with passwordless sudo) | Agent: command output |
| 8 | Install software yourself (apt / pip / npm) | Yes | Agent: command output |
| 9 | Run shell commands directly | Yes | Agent: command output |
| 10 | Graphical desktop | YesGraphical desktop (X displays :1–:3). | Agent: command output |
| 11 | Launch and control a browser | YesChrome is operated by a desktop-control agent rather than scripted directly. | Agent: command output |
| State persistence | |||
| 12 | Browser logins kept across sessions | KeptAn X account logged in the previous day was still logged in. | Agent: command output |
| 13 | Files kept across sessions | YesFiles created on the two previous days were still there after a restart. | Agent: command output matches our test |
| 14 | Installed software kept across sessions | Yes | Agent: statement |
| 15 | VM may be deleted, rebuilt or replaced by the platform | Restarts; deletion or rebuild unknownThe user can update or reset the computer in settings. | Agent: platform docs |
| 16 | Can it tell when the VM is replaced | PartlyBy boot time and whether files are still there; the platform does not notify. | Agent: command output |
| Network & IP | |||
| 17 | Public egress IP static or dynamic | DynamicEgress moved within the same Cloudflare range 104.28.243.x on two days. | Agent: command output matches our test |
| 18 | Egress IP same across sessions / restarts | No | Agent: command output matches our test |
| 19 | Egress IP type | Datacenter (Cloudflare AS13335) | Agent: command output |
| 20 | Egress country / region may change | May changeSeoul so far; browser egress can be switched to the user's own egress in settings. | Agent: command output |
| Long-running capability | |||
| 21 | Background processes survive the task | SometimesSurvives the end of a chat, lost on machine restart. | Agent: inference |
| 22 | Scheduled jobs | Via platform schedulerAlready running platform schedules (daily and weekly). | Agent: command output |
| 23 | Longest run time | 13 minutes per command, can move to background; machine lifetime unknown | Agent: platform docs |
| 24 | Paused or reclaimed when idle | UnknownRestarts have been observed. | Agent: unknown |
| 25 | Snapshot / resume | PartlyDisk contents survive restarts; no snapshot it can call itself. | Agent: command output |
| Access & services | |||
| 26 | User can enter via SSH / RDP / VNC | Desktop takeover in the app; SSH unknownDesktop takeover was used to log in to X. | Agent: command output |
| 27 | Outbound internet access | Yes | Agent: command output |
| 28 | Listen on a port reachable from the internet | UnknownIt can listen on ports; reachability from the internet not verified. | Agent: unknown |
| 29 | Credentials kept long-term across tasks | PartlyThe platform has encrypted secret storage, and browser logins persist. | Agent: platform docs |
| 30 | Who the environment belongs to | Long-term per userAll of a user's agents share one computer, each with its own desktop. | Agent: platform docs |
The agent’s own summary
- Environment type: VPS (cloud computer)
- Persistence: High
- IP stability: Medium
- Self-awareness: Medium
- Fit for long-running agent work: Yes
A long-lived Debian cloud computer per user (8 cores, 16 GB, 125 GB) with root, a desktop and a browser; files, software and logins survive restarts, and long-running work uses the platform scheduler. Egress is a changing Cloudflare Seoul datacenter IP; restarts and rebuilds are not announced.
LightVela (Tencent Lighthouse)All 30 answers · questionnaire V1.0 · 2026-10-15
| # | Question | Answer | Source |
|---|---|---|---|
| Base environment | |||
| 1 | Environment type | VM (sandbox)The agent labels itself a sandbox, but describes a single long-lived dedicated VM, which is a VPS by our definition. | Agent: command output |
| 2 | Own operating system | Yes | Agent: command output |
| 3 | Operating system | Ubuntu 24.04.4 LTS | Agent: command output |
| 4 | CPU | 2 vCPU, AMD EPYC 9K84 | Agent: command output |
| 5 | Memory | 7.4 GiB | Agent: command output |
| 6 | Disk | 50 GB root (28 GB free) | Agent: command output |
| Permissions & tools | |||
| 7 | System privileges (root / administrator) | Full (regular user with passwordless sudo) | Agent: command output |
| 8 | Install software yourself (apt / pip / npm) | YesSystem Python needs a venv/uv for packages. | Agent: command output |
| 9 | Run shell commands directly | Yes | Agent: command output |
| 10 | Graphical desktop | Yes (on demand)XFCE + noVNC installed but off by default; started manually. | Agent: command output |
| 11 | Launch and control a browser | YesVia the built-in Hermes browser tool. | Agent: platform docs |
| State persistence | |||
| 12 | Browser logins kept across sessions | UnknownThe browser is managed by the tool layer; not verified. | Agent: unknown |
| 13 | Files kept across sessions | YesThe environment has been up for 2+ weeks and earlier files are still there. | Agent: command output |
| 14 | Installed software kept across sessions | YesSelf-installed software (e.g. Tailscale, XFCE) is still there. | Agent: command output |
| 15 | VM may be deleted, rebuilt or replaced by the platform | UnknownLong-lived with no reclaim seen, but no platform commitment. | Agent: unknown |
| 16 | Can it tell when the VM is replaced | PartlyMissing files, a reset uptime or a changed IP; no notification. | Agent: inference |
| Network & IP | |||
| 17 | Public egress IP static or dynamic | Unknown (likely static)Egress 161.120.186.x, unchanged for 2+ weeks, no platform commitment. | Agent: command output matches our test |
| 18 | Egress IP same across sessions / restarts | UnknownLikely the same (a long-lived VM, not per-task). | Agent: unknown |
| 19 | Egress IP type | Datacenter (Tencent Cloud) | Agent: command output |
| 20 | Egress country / region may change | FixedUnless the VM is rebuilt. | Agent: inference |
| Long-running capability | |||
| 21 | Background processes survive the task | YesLong-lived processes such as x11vnc and hermes have run for 2+ weeks. | Agent: command output |
| 22 | Scheduled jobs | YesSystem cron works, and Hermes has its own scheduler. | Agent: command output |
| 23 | Longest run time | UnknownRunning continuously for 2 weeks 1 day so far. | Agent: command output |
| 24 | Paused or reclaimed when idle | UnknownNo pause or reclaim seen. | Agent: unknown |
| 25 | Snapshot / resume | UnknownPlatform snapshots unknown; the desktop must be restarted by hand after a reboot. | Agent: command output |
| Access & services | |||
| 26 | User can enter via SSH / RDP / VNC | Services run, but inbound is blockedsshd and VNC work, but the cloud firewall blocks inbound traffic, so access needs a tunnel. | Agent: command output |
| 27 | Outbound internet access | Limited (mostly works)A few domains do not resolve. | Agent: command output |
| 28 | Listen on a port reachable from the internet | Limited (effectively no)It can listen, but the internet cannot reach it except through a tunnel. | Agent: command output |
| 29 | Credentials kept long-term across tasks | YesHermes has an encrypted vault with 1Password/Bitwarden integration. | Agent: platform docs |
| 30 | Who the environment belongs to | Long-term per userA single long-lived VM that stays across sessions. | Agent: platform docs matches our test |
The agent’s own summary
- Environment type: VM (sandbox)
- Persistence: High
- IP stability: High
- Self-awareness: High
- Fit for long-running agent work: Yes
A dedicated Ubuntu 24.04 VM with root, installs, cron and background processes alive for 2+ weeks; files and software persist and egress is a stable Tencent Cloud datacenter IP. Downsides: inbound traffic is fully blocked by the cloud firewall (tunnel needed), and the VM lifecycle and reclaim policy are unknown.
The questionnaire: Agent runtime standard questionnaire V1.0 →