KeysUsageBilling & Credits
Guides
HomeAPIStatus

Video Generation

Suggest Edits

Installation

pip install lumaai

https://pypi.org/project/lumaai/

Authentication

  1. Get a key from https://lumalabs.ai/dream-machine/api/keys
  2. Pass it to client sdk by either
    1. setting LUMAAI_API_KEY
    2. or passing auth_token to the client

Setting up client

Using LUMAAI_API_KEY env variable

from lumaai import LumaAI

client = LumaAI()

Using auth_token parameter

import os
from lumaai import LumaAI

client = LumaAI(
    auth_token=os.environ.get("LUMAAI_API_KEY"),
)

How do I get the video for a generation?

  • Right now the only supported way is via polling
  • The create endpoint returns an id which is an UUID V4
  • You can use it to poll for updates (you can see the video at generation.assets.video)

Usage Example

import requests
import time
from lumaai import LumaAI

client = LumaAI()

generation = client.generations.create(
  prompt="A teddy bear in sunglasses playing electric guitar and dancing",
)
completed = False
while not completed:
  generation = client.generations.get(id=generation.id)
  if generation.state == "completed":
    completed = True
  elif generation.state == "failed":
    raise RuntimeError(f"Generation failed: {generation.failure_reason}")
  print("Dreaming")
  time.sleep(3)

video_url = generation.assets.video

# download the video
response = requests.get(video_url, stream=True)
with open(f'{generation.id}.mp4', 'wb') as file:
    file.write(response.content)
print(f"File downloaded as {generation.id}.mp4")

Async library

Import and use AsyncLumaai

import os
from lumaai import AsyncLumaAI

client = AsyncLumaAI(
    auth_token=os.environ.get("LUMAAI_API_KEY"),
)

For all the functions add await (eg. below)

generation = await client.generations.create(
    prompt="A teddy bear in sunglasses playing electric guitar and dancing",
)

Text to Video

generation = client.generations.create(
    prompt="A teddy bear in sunglasses playing electric guitar and dancing",
)

Downloading a video

import requests

url = 'https://example.com/video.mp4'
response = requests.get(url, stream=True)

file_name = 'video.mp4'
with open('video.mp4', 'wb') as file:
    file.write(response.content)
print(f"File downloaded as {file_name}")

With loop, aspect ratio

generation = client.generations.create(
    prompt="A teddy bear in sunglasses playing electric guitar and dancing",
    loop=True,
    aspect_ratio="3:4"
)

Image to Video

☁️

Image URL

You should upload and use your own cdn image urls, currently this is the only way to pass an image

With start frame

generation = client.generations.create(
    prompt="Low-angle shot of a majestic tiger prowling through a snowy landscape, leaving paw prints on the white blanket",
    keyframes={
      "frame0": {
        "type": "image",
        "url": "https://storage.cdn-luma.com/dream_machine/7e4fe07f-1dfd-4921-bc97-4bcf5adea39a/video_0_thumb.jpg"
      }
    }
)

With start frame, loop

generation = client.generations.create(
    prompt="Low-angle shot of a majestic tiger prowling through a snowy landscape, leaving paw prints on the white blanket",
    loop=True,
    keyframes={
      "frame0": {
        "type": "image",
        "url": "https://storage.cdn-luma.com/dream_machine/7e4fe07f-1dfd-4921-bc97-4bcf5adea39a/video_0_thumb.jpg"
      }
    }
)

With ending frame

generation = client.generations.create(
    prompt="Low-angle shot of a majestic tiger prowling through a snowy landscape, leaving paw prints on the white blanket",
    keyframes={
      "frame1": {
        "type": "image",
        "url": "https://storage.cdn-luma.com/dream_machine/7e4fe07f-1dfd-4921-bc97-4bcf5adea39a/video_0_thumb.jpg"
      }
    }
)

With start and end keyframes

generation = client.generations.create(
    prompt="Low-angle shot of a majestic tiger prowling through a snowy landscape, leaving paw prints on the white blanket",
    keyframes={
      "frame0": {
        "type": "image",
        "url": "https://storage.cdn-luma.com/dream_machine/7e4fe07f-1dfd-4921-bc97-4bcf5adea39a/video_0_thumb.jpg"
      },
      "frame1": {
        "type": "image",
        "url": "https://storage.cdn-luma.com/dream_machine/12d17326-a7b6-4538-b9b7-4a2e146d4488/video_0_thumb.jpg"
      }
    }
)

Extend Video

Extend video

Extend is currently supported only for generated videos. Please make sure the generation is in completed state before passing it

generation = client.generations.create(
    prompt="A teddy bear in sunglasses playing electric guitar and dancing",
    keyframes={
      "frame0": {
        "type": "generation",
        "id": "d1968551-6113-4b46-b567-09210c2e79b0"
      }
    }
)

Reverse extend video

Generate video leading up to the provided video.

Extend is currently supported only for generated videos. Please make sure the generation is in completed state before passing it

generation = client.generations.create(
    prompt="A teddy bear in sunglasses playing electric guitar and dancing",
    keyframes={
      "frame1": {
        "type": "generation",
        "id": "d1968551-6113-4b46-b567-09210c2e79b0"
      }
    }
)

Extend a video with an end-frame

Extend is currently supported only for generated videos. Please make sure the generation is in completed state before passing it

generation = client.generations.create(
    prompt="Low-angle shot of a majestic tiger prowling through a snowy landscape, leaving paw prints on the white blanket",
    keyframes={
      "frame0": {
        "type": "generation",
        "id": "d1968551-6113-4b46-b567-09210c2e79b0"
      },
      "frame1": {
        "type": "image",
        "url": "https://storage.cdn-luma.com/dream_machine/12d17326-a7b6-4538-b9b7-4a2e146d4488/video_0_thumb.jpg"
      }
    }
)

Reverse extend a video with a start-frame

Extend is currently supported only for generated videos. Please make sure the generation is in completed state before passing it

generation = client.generations.create(
    prompt="Low-angle shot of a majestic tiger prowling through a snowy landscape, leaving paw prints on the white blanket",
    keyframes={
      "frame0": {
        "type": "image",
        "url": "https://storage.cdn-luma.com/dream_machine/12d17326-a7b6-4538-b9b7-4a2e146d4488/video_0_thumb.jpg"
      },
      "frame1": {
        "type": "generation",
        "id": "d1968551-6113-4b46-b567-09210c2e79b0"
      }
    }
)

Interpolate between 2 videos

Interpolate is currently supported only for generated videos. Please make sure the generation is in completed state before passing it

generation = client.generations.create(
    prompt="A teddy bear in sunglasses playing electric guitar and dancing",
    keyframes={
      "frame1": {
        "type": "generation",
        "id": "d312d37a-7ff4-49f2-94f8-218f3fe2a4bd"
      },
      "frame1": {
        "type": "generation",
        "id": "d1968551-6113-4b46-b567-09210c2e79b0"
      }
    }
)

Generations

Get generation with id

generation = client.generations.get(id="d1968551-6113-4b46-b567-09210c2e79b0")

List all generations

generation = client.generations.list(limit=100, offset=0)

Delete generation

generation = client.generations.delete(id="d1968551-6113-4b46-b567-09210c2e79b0")

Camera Motions

📘

How to use camera motion

Just add the camera motion value as part of prompt itself

Get all supported camera motions

supported_camera_motions = client.generations.camera_motion.list()

How to use camera motion

Camera is controlled by language in Dream Machine. You can find supported camera moves by calling the Camera Motions endpoint. This will return an array of supported camera motion strings (like "camera orbit left") which can be used in prompts. In addition to these exact strings, syntactically similar phrases also work, though there can be mismatches sometimes.


How to get a callback when generation has an update

  • It will get status updates (dreaming/completed/failed)
  • It will also get the video url as part of it when completed
  • It's a POST endpoint you can pass, and request body will have the generation object in it
  • It expected to be called multiple times for a status
  • If the endpoint returns a status code other than 200, it will be retried max 3 times with 100ms delay and the request has a 5s timeout

example

generation = await client.generations.create(
    prompt="A teddy bear in sunglasses playing electric guitar and dancing",
    callback_url="<your_api_endpoint_here>"
)

Updated 24 days ago