ClimateIQ AI – 11 Mux Features Power Climate Action for 2B+ Users

This is a submission for the DEV's Worldwide Show and Tell Challenge Presented by Mux

What I Built

ClimateIQ AI is a production-grade climate intelligence platform transforming satellite data into actionable insights for farmers, cities, and communities worldwide.

🌍 The Full Platform (Beyond Video)

ClimateIQ already has deep climate features:

Tech Stack: Next.js 14, TypeScript, Supabase, Google Gemini AI, Mapbox, NASA FIRMS, NREL PVWatts, Planet API, SoilGrids

📹 NEW: Community Hub (Powered by Mux)

The missing piece was community-generated content. Rural communities couldn't share drought videos. Cities couldn't broadcast awareness campaigns. Climate educators needed multilingual reach.

Mux solved this with 11 integrated features:

• 📹 Climate Stories – Upload video testimonials with auto-generated captions
• 🌍 Climate Live – Low-latency streaming for real-time expert consultations
• 🌐 AI Translation – Captions in Hindi, Spanish, French for global reach
• 🎙️ Audio Dubbing – ElevenLabs voice cloning + Mux multi-track audio

My Pitch Video

Demo

🔗 Live Demo: https://climate-iq-git-feature-omkardongres-projects.vercel.app/

📦 GitHub: https://github.com/omkardongre/climate-iq/tree/feature

Test Credentials:

• Email: test@gmail.com
• Password: 12345678

🔓 Without Login: View videos, watch with captions, switch dubbed audio tracks, watch live streams

🔐 With Login: Upload videos, generate chapters, translate captions, dub audio, go live

How to Test Mux Features (All from the UI)

1. Upload Video: Go to Community Hub → Click "Share Your Story" button → Select a video → Watch it upload with progress bar
2. AI Chapters: After upload completes, click on your video → Click "Generate AI Chapters" button → See chapters appear below the player
3. Translate Captions: Click "Translate Captions" button → Select Hindi/Spanish from dropdown → After ~40 seconds, check the CC menu in the video player
4. Dub Audio: Click "Dub Audio" button → Select language → Wait ~2 minutes → Check the Audio/Language menu in the player settings (gear icon)
5. Live Streaming: Click "Go Live" button → Copy the RTMP URL and Stream Key → Use OBS or any streaming software to broadcast

The Story Behind It

The Climate Data Gap: Satellites collect terabytes of climate data daily. But farmers facing crop failures, cities battling pollution, and communities seeking education can't access it. ClimateIQ bridges this gap.

Why Video Was Missing:

• A community in India showing drought damage creates more impact than statistics
• Live broadcasts enable real-time expert consultations during crises
• 2B+ potential users need content in their language

Why Mux:

• We needed auto-captions for accessibility
• Low-latency for real-time consultations
• Multi-language support for global reach
• Production-ready infrastructure in days, not months

Technical Highlights

11 Mux Features Integrated:

Use of Mux (Additional Prize Category)

Mux isn't just hosting videos – it's the backbone of our entire community feature.

🎬 Beyond Basic Hosting

1. AI-Powered Chapters (@mux/ai SDK)

import { generateChapters } from '@mux/ai/workflows';
const chapters = await generateChapters(assetId, 'en', { provider: 'google' });

Climate videos get auto-chapters like "Impact on Agriculture" or "Mitigation Strategies" – helping users navigate long content.

2. Multi-Track Audio Dubbing

await muxVideo.assets.createTrack(assetId, {
  type: 'audio',
  url: supabaseAudioUrl,
  language_code: 'hi',
  name: 'AI Dubbed (hi)'
});

ElevenLabs generates dubbed audio → we add it to the Mux asset → users switch languages natively in the player.

3. Low-Latency Live Streaming

await muxVideo.liveStreams.create({
  reduced_latency: true,
  reconnect_window: 60,
  generated_subtitles: [{ language_code: 'en', name: 'English (auto)' }]
});

Users broadcast with low-latency. Experts provide real-time advice during live sessions.

4. Auto-Captions + AI Translation Pipeline

1. Upload with generated_subtitles → Mux generates English captions
2. Fetch VTT from https://stream.mux.com/{playbackId}/text/{trackId}.vtt
3. Gemini translates to Hindi/Spanish
4. Upload to Supabase + add via assets.createTrack()

📊 What Makes This Deep Integration

We're not just hosting videos. We're using:

• Official @mux/ai SDK for chapter generation
• Multi-Track Audio for dubbed languages
• Text Track API for translated captions
• Master Access for AI processing pipelines
• Low-Latency Live Streaming for real-time advice
• Custom Analytics Dimensions for climate insights

🌍 Real Impact

Mux handles encoding, CDN, multi-track, and AI – so we focus on climate impact.

❗ By submitting this project, I confirm that my video adheres to Mux's terms of service: https://www.mux.com/terms