Skip to main content

Data Formats & Communication Protocol

Overview​

The YouTube download toolkit uses a simple yet powerful text-based communication protocol to transfer video collection data from the Chrome extension to the Java application. This protocol is designed to be human-readable, easily parseable, and flexible enough to handle various categorization and quality preferences.

Core Data Format​

Basic Structure​

The communication format consists of two types of lines:

  1. Parameter Lines: Define settings that apply to subsequent video URLs
  2. URL Lines: YouTube video links to be downloaded
f=category_name;q=quality_setting
https://www.youtube.com/watch?v=VIDEO_ID
https://www.youtube.com/watch?v=ANOTHER_VIDEO_ID

Parameter Syntax​

Parameters use a key-value format separated by semicolons:

f=folder_name;q=quality_level

Parameter Keys:

  • f = Folder/Category: Specifies the destination folder name
  • q = Quality: Defines the video quality preference

Quality Values:

  • 360 = 360p resolution
  • 480 = 480p resolution
  • 720 = 720p resolution (HD)
  • 1080 = 1080p resolution (Full HD)
  • best = Highest available quality (fastest download)

Data Flow Examples​

Example 1: Simple Categorization​

f=music;q=720
https://www.youtube.com/watch?v=dQw4w9WgXcQ
https://www.youtube.com/watch?v=oHg5SJYRHA0
f=tutorials;q=1080
https://www.youtube.com/watch?v=kJQP7kiw5Fk

Result:

  • Two music videos downloaded in 720p quality to /music/ folder
  • One tutorial video downloaded in 1080p quality to /tutorials/ folder

Example 2: Mixed Quality Settings​

f=entertainment;q=480
https://www.youtube.com/watch?v=VIDEO1
https://www.youtube.com/watch?v=VIDEO2
f=education;q=best
https://www.youtube.com/watch?v=VIDEO3

Result:

  • Entertainment videos in 480p (smaller file sizes)
  • Educational content in best available quality

Example 3: Inline Parameters​

Individual videos can override global settings using inline parameters:

f=default;q=720
https://www.youtube.com/watch?v=VIDEO1
https://www.youtube.com/watch?v=VIDEO2;f=special;q=1080
https://www.youtube.com/watch?v=VIDEO3

Result:

  • VIDEO1 and VIDEO3: 720p in /default/ folder
  • VIDEO2: 1080p in /special/ folder (overrides global settings)

Chrome Extension Data Generation​

Internal Storage Format​

The Chrome extension stores collected data in a structured JavaScript object:

{
  "music": [
    "https://www.youtube.com/watch?v=dQw4w9WgXcQ",
    "https://www.youtube.com/watch?v=oHg5SJYRHA0"
  ],
  "tutorials": [
    "https://www.youtube.com/watch?v=kJQP7kiw5Fk"
  ]
}

Export Transformation​

The popup interface converts the internal format to the communication protocol:

const newText = Object.entries(listObj)
  .sort(([keyA], [keyB]) => keyA.localeCompare(keyB))
  .map(([key, value]) => `${key}\n${value.filter(Boolean).join('\n')}`)
  .join('\n');

Transformation Process:

  1. Alphabetical Sorting: Categories are sorted for consistent output
  2. Header Generation: Each category becomes a parameter line
  3. URL Listing: Videos are listed under their respective categories
  4. Deduplication: Empty or duplicate entries are filtered out

Java Application Data Processing​

URL Parsing Logic​

The Java application uses the Video.fromURL() method to extract parameters:

public static Video fromURL(String url) {
    String[] arr = url.split("\\?");
    Map<String, String> params = getParams(arr[1]);
    String folder = params.get("f");
    String quality = params.get("q");
    String id = params.get("v");
    return new Video(id, folder, quality);
}

Parameter Processing​

The getParams() method handles both semicolon and ampersand separators:

public static Map<String, String> getParams(String p) {
    Map<String, String> params = new HashMap<>();
    String[] pairs = p.split(";|&");  // Supports both ; and & separators
    for (String pair : pairs) {
        String[] keyValue = pair.split("=");
        if (keyValue.length == 2) {
            params.put(keyValue[0], keyValue[1]);
        }
    }
    return params;
}

File Processing Workflow​

The application processes input files line by line:

for(String link : links){
    if(link.startsWith("f=")){
        presetParams(link);  // Set global parameters
    } else if(link.startsWith("https://www.youtube.com/watch?v")){
        checkAndDownloadVideoMP4(link, resolution, false);  // Download video
    }
}

Advanced Features​

Parameter Inheritance​

Parameters set by header lines apply to all subsequent URLs until a new header is encountered:

f=music;q=360          # Global setting
https://www.youtube.com/watch?v=VIDEO1    # Uses music/360p
https://www.youtube.com/watch?v=VIDEO2    # Uses music/360p
f=podcasts;q=best      # New global setting
https://www.youtube.com/watch?v=VIDEO3    # Uses podcasts/best

URL Normalization​

The system handles various YouTube URL formats:

Input Formats:

  • https://www.youtube.com/watch?v=VIDEO_ID&list=PLAYLIST
  • https://www.youtube.com/watch?v=VIDEO_ID&t=120s
  • https://youtu.be/VIDEO_ID

Normalized Output:

  • https://www.youtube.com/watch?v=VIDEO_ID

Quality Fallback Logic​

When a specific quality isn't available, the Java application implements intelligent fallback:

  1. Exact Match: Try to find the requested resolution
  2. Best Available: Select highest quality if exact match unavailable
  3. Format Combination: Combine best video + best audio streams
  4. Simple Fallback: Use youtube-dl's "best" format as last resort

Error Handling​

Malformed Data Recovery​

The system gracefully handles various error conditions:

Invalid Parameter Lines:

invalid_parameter_line
f=music;q=720  # Valid line processed normally

Missing Parameters:

f=music        # Missing quality - uses default
;q=720         # Missing folder - uses default

Invalid URLs:

not_a_youtube_url
https://www.youtube.com/watch?v=INVALID_ID

Data Validation​

Both components implement validation:

Chrome Extension:

  • Filters empty lines and invalid entries
  • Validates YouTube URL format before storage
  • Deduplicates video IDs automatically

Java Application:

  • Skips malformed lines with error messages
  • Validates video ID format (11 characters)
  • Provides fallback for missing parameters

File Format Specifications​

List File Format (.txt)​

MIME Type: text/plain Encoding: UTF-8 Line Endings: Platform-specific (\n or \r\n)

Structure:

[parameter_line]
[url_line]*
[parameter_line]
[url_line]*
...

Configuration Integration​

The protocol integrates with the Java application's configuration system:

Global Defaults:

  • destinationFolderName: Base directory for all downloads
  • downloadedFileName: Template for file naming
  • defaultListFile: Default input file path

Runtime Overrides:

  • Parameter lines override global folder settings
  • Inline URL parameters override both global and header settings
  • Command-line resolution input overrides file-specified quality

This communication protocol provides a robust, flexible, and user-friendly way to transfer complex video collection data between the Chrome extension and Java application while maintaining simplicity and human readability.