The Skylight Windows VM supports various remote interaction methods that let you control the instance programmatically.

Available Operations

  • click - Perform a mouse click
  • drag - Perform a drag operation
  • screenshot - Take a screenshot
  • move - Move the mouse cursor
  • keypress - Press keys simultaneously
  • type - Type text
  • scroll - Scroll the screen
  • file - Get a file from the VM
  • install - Install applications

Methods

click

Perform a click operation at specified coordinates or current position.

ParameterTypeRequiredDefaultDescription
xinteger | nullNonullX coordinate to click at
yinteger | nullNonullY coordinate to click at
buttonenumNo"left"Mouse button to use ("left", "right", "middle")
clicksintegerNo1Number of clicks (range: 1-10)
intervalnumber | nullNo0Time between clicks in seconds (range: 0-1)
# Simple left click at coordinates
skylight.interact.click(instance_id="instance123", x=300, y=400)

# Right click
skylight.interact.click(instance_id="instance123", x=300, y=400, button="right")

# Middle click
skylight.interact.click(instance_id="instance123", button="middle")

# Double click
skylight.interact.click(instance_id="instance123", x=300, y=400, clicks=2)

# Click at current position (no x,y specified)
skylight.interact.click(instance_id="instance123")

# Click with 0.5 second interval between clicks
skylight.interact.click(instance_id="instance123", x=300, y=400, clicks=2, interval=0.5)

drag

Perform a drag operation along a path of coordinates.

ParameterTypeRequiredDefaultDescription
pathobject[]Yes-List of points to drag through, each with x and y coordinates
buttonenumNo"left"Mouse button to use ("left", "right", "middle")
stepinteger | nullNonullStep size for drag movements (range: 1-100)
# Drag from one point to another
skylight.interact.drag(instance_id="instance123", path=[
    {"x": 100, "y": 200},
    {"x": 300, "y": 400}
])

# Specify step size for more granular movement
skylight.interact.drag(instance_id="instance123", path=[
    {"x": 100, "y": 200},
    {"x": 300, "y": 400}
], step=10)

# Use a different mouse button
skylight.interact.drag(instance_id="instance123", path=[
    {"x": 100, "y": 200},
    {"x": 300, "y": 400}
], button="right")

screenshot

Take a screenshot of the VM.

# Take a screenshot
screenshot = skylight.interact.screenshot(instance_id="instance123")
# Returns a base64 encoded image that can be saved or processed

move

Move the mouse to the specified coordinates.

ParameterTypeRequiredDescription
xintegerYesX coordinate to move to
yintegerYesY coordinate to move to
# Move mouse to coordinates
skylight.interact.move(instance_id="instance123", x=100, y=200)

keypress

Press the specified keys simultaneously.

ParameterTypeRequiredDescription
keysstring[]YesList of keys to press simultaneously
# Press Enter key
skylight.interact.keypress(instance_id="instance123", keys=["Enter"])

# Press Ctrl+C
skylight.interact.keypress(instance_id="instance123", keys=["ctrl", "c"])

# Press Ctrl+Alt+Delete
skylight.interact.keypress(instance_id="instance123", keys=["ctrl", "alt", "delete"])

# Take a screenshot with Windows+Shift+S
skylight.interact.keypress(instance_id="instance123", keys=["super", "shift", "s"])

type

Type the specified text with optional delay between keystrokes.

ParameterTypeRequiredDefaultDescription
textstringYes-Text to type
intervalnumber | nullNo0.05Time between keystrokes in seconds (range: 0.01-0.5)
# Type text with default interval between keystrokes
skylight.interact.type(instance_id="instance123", text="Hello World")

# Type with custom interval between keystrokes (in seconds)
skylight.interact.type(instance_id="instance123", text="Hello World", interval=0.1)

scroll

Scroll at the specified coordinates.

ParameterTypeRequiredDefaultDescription
xintegerYes-X coordinate to scroll at
yintegerYes-Y coordinate to scroll at
scroll_xintegerNo0Horizontal scroll amount
scroll_yintegerNo0Vertical scroll amount
stepinteger | nullNo-Step size for scroll movements (range: 1-100)
# Scroll down
skylight.interact.scroll(instance_id="instance123", x=400, y=300, scroll_y=100)

# Scroll up
skylight.interact.scroll(instance_id="instance123", x=400, y=300, scroll_y=-100)

# Scroll right
skylight.interact.scroll(instance_id="instance123", x=400, y=300, scroll_x=100)

# Scroll left
skylight.interact.scroll(instance_id="instance123", x=400, y=300, scroll_x=-100)

# Scroll with steps
skylight.interact.scroll(instance_id="instance123", x=400, y=300, scroll_y=100, step=5)

file

Get a secure download link for a file on the VM.

ParameterTypeRequiredDescription
pathstringYesPath to the file on the VM
# Get a file from the VM
download_url = skylight.interact.get_file(instance_id="instance123", request_body={"key": "/path/to/file.txt"})
# Returns a presigned URL to download the file

install

Install Chocolatey packages on a Windows instance.

ParameterTypeRequiredDescription
packagesstring[]YesList of Chocolatey package names to install
# Install packages
result = skylight.interact.install(instance_id="instance123", packages=["googlechrome", "vscode"])

Practical Examples

Opening an Application

# Move mouse to Start button and click
skylight.interact.click(instance_id="instance123", x=20, y=980)

# Type the application name
skylight.interact.type(instance_id="instance123", text="notepad")

# Press Enter to launch
skylight.interact.keypress(instance_id="instance123", keys=["Enter"])

Drag and Drop Operation

# Select a file by clicking on it
skylight.interact.click(instance_id="instance123", x=200, y=300)

# Drag the file to a new location
skylight.interact.drag(instance_id="instance123", path=[
    {"x": 200, "y": 300},  # Starting position (file location)
    {"x": 500, "y": 400}   # Ending position (destination)
])

Working with Text

# Click to position cursor
skylight.interact.click(instance_id="instance123", x=300, y=400)

# Type some text
skylight.interact.type(instance_id="instance123", text="This is some sample text.")

# Select all text (Ctrl+A)
skylight.interact.keypress(instance_id="instance123", keys=["ctrl", "a"])

# Copy text (Ctrl+C)
skylight.interact.keypress(instance_id="instance123", keys=["ctrl", "c"])

# Click somewhere else
skylight.interact.click(instance_id="instance123", x=300, y=500)

# Paste text (Ctrl+V)
skylight.interact.keypress(instance_id="instance123", keys=["ctrl", "v"])

Taking Screenshots and Saving Files

# Take a screenshot
screenshot = skylight.interact.screenshot(instance_id="instance123")

# Save the screenshot locally (assuming base64 encoding)
import base64
with open("screenshot.png", "wb") as f:
    f.write(base64.b64decode(screenshot))

# Get a file from the VM
download_url = skylight.interact.get_file(instance_id="instance123", request_body={"key": "C:/Users/Administrator/Documents/example.txt"})

# Download the file using the URL
import requests
response = requests.get(download_url)
with open("example.txt", "wb") as f:
    f.write(response.content)