Build Week #1, Day 2:
AI documents processing workflows

The challenges of processing documents with AI

Processing documents with AI involves compute-intensive and long-running processes dealing with multiple possible failures along the way:

• Each step of the workflow can face network issues or instabilities
• OpenAI Whisper 20MB file upload limit requires to split large files into smaller ones
• The processing of large amounts of files needs to be parallelized while matching OpenAI's account rate limit

Defer workflow's pattern helps with dividing the whole process into smaller sub-tasks that get specific rate limits and retry configuration, for optimal processing power.

Today's Build Week demo showcases a workflow consuming large meeting video files to extract audio as text using OpenAI Whisper.

• The entry point of the workflow is a daily CRON that triggers a sub-task for each available meeting recording.
• Then, each meeting video is split into small .mp3 files and sent to another sub-tasks that upload it to OpenAI Whisper API.

This Defer workflow optimizes the parallelization of the video processing while matching OpenAI Whisper's 20MB file upload limit and mutualizing OpenAI's account rate limit:

Deploy this AI workflow in one click

The above meeting video transcript workflow is available on GitHub with one-click deployment to Vercel.