Skip to main content

JSON Processing Utilities

Overview​

The JSON processing utilities provide essential functions for parsing, filtering, sorting, and manipulating JSON data within the application. These functions handle the core data operations that power the visualizer's search, display, and navigation features.

Key Extraction (extractKeys)​

export const extractKeys = (data: JSONObject[]): string[] => {
  const keys = new Set<string>();
  
  const processObject = (obj: JSONObject, prefix = '') => {
    for (const [key, value] of Object.entries(obj)) {
      // Skip internal properties
      if (key.startsWith('__')) continue;
      
      const currentKey = prefix ? `${prefix}.${key}` : key;
      keys.add(currentKey);
      
      // Process nested objects
      if (value !== null && typeof value === 'object' && !Array.isArray(value)) {
        processObject(value as JSONObject, currentKey);
      }
    }
  };
  
  data.forEach(obj => processObject(obj));
  return Array.from(keys);
};

Purpose: Extracts all possible field names from a dataset, including nested object properties.

How it works:

  1. Uses a Set to automatically deduplicate field names
  2. Recursive processing - The processObject function calls itself for nested objects
  3. Dot notation - Nested fields are represented as parent.child.grandchild
  4. Skips internal fields - Properties starting with __ (like __id, __flags) are ignored
  5. Type checking - Only processes actual objects, not arrays or primitive values

Example:

const data = [
  { name: "John", address: { street: "123 Main St", city: "Boston" } },
  { name: "Jane", phone: "555-1234" }
];

extractKeys(data); 
// Returns: ["name", "address", "address.street", "address.city", "phone"]

Value Formatting (formatValue)​

export const formatValue = (value: JSONValue): string => {
  if (value === null) return 'null';
  if (value === undefined) return 'undefined';
  if (typeof value === 'object') return JSON.stringify(value);
  return String(value);
};

Purpose: Converts any JSON value to a string for display purposes.

Handling different types:

  • null/undefined - Returns literal strings "null" and "undefined"
  • Objects/Arrays - Converts to JSON string representation
  • Primitives - Converts to string using String() constructor

JSON Parsing (parseJson)​

export const parseJson = (jsonString: string): JSONObject[] | null => {
  try {
    const parsed = JSON.parse(jsonString);
    if (Array.isArray(parsed)) {
      return parsed;
    } else if (typeof parsed === 'object' && parsed !== null) {
      return [parsed];
    }
    return null;
  } catch (error) {
    console.error('Failed to parse JSON:', error);
    return null;
  }
};

Purpose: Safely parses JSON strings and normalizes the result to an array format.

Normalization logic:

  • Array input - Returns as-is
  • Object input - Wraps in an array [object]
  • Other types - Returns null (invalid)
  • Parse errors - Returns null and logs error

This ensures the application always works with arrays of objects, regardless of input format.

Path-based Value Access (getValueByPath)​

export const getValueByPath = (obj: JSONObject, path: string): JSONValue | undefined => {
  const keys = path.split('.');
  return keys.reduce<unknown>((o, key) => {
    if (o && typeof o === 'object' && key in o) {
      return (o as Record<string, unknown>)[key];
    }
    return undefined;
  }, obj);
};

Purpose: Retrieves values from nested objects using dot notation paths.

How it works:

  1. Splits path - "address.street" becomes ["address", "street"]
  2. Reduces through keys - Traverses the object one key at a time
  3. Type safety - Checks that each step is an object before accessing properties
  4. Graceful failure - Returns undefined if any part of the path doesn't exist

Example:

const obj = { user: { profile: { name: "John" } } };
getValueByPath(obj, "user.profile.name"); // Returns: "John"
getValueByPath(obj, "user.profile.age");  // Returns: undefined

Object Expansion Check (isExpandableObject)​

export const isExpandableObject = (value: JSONValue): boolean => {
  return (
    value !== null && 
    typeof value === 'object' && 
    !Array.isArray(value) && 
    Object.keys(value).length > 0
  );
};

Purpose: Determines if a value is a non-empty object that can be expanded in the UI.

Criteria for expandable objects:

  • Not null
  • Is an object type
  • Not an array
  • Has at least one property

Data Filtering (filterData)​

export const filterData = (data: JSONObject[], searchTerm: string): JSONObject[] => {
  if (!searchTerm.trim()) return data;
  
  let regex: RegExp;
  try {
    // Try to use the search term as a regex pattern
    regex = new RegExp(searchTerm, 'i');
  } catch (error) {
    // If invalid regex, use as literal string
    regex = new RegExp(searchTerm.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'), 'i');
  }
  
  return data.filter(item => searchInObject(item, regex));
};

Purpose: Filters data based on a search term with automatic regex support.

Smart regex handling:

  1. Attempts regex - Tries to use search term as a regular expression
  2. Fallback to literal - If regex is invalid, escapes special characters for literal matching
  3. Case insensitive - Uses the 'i' flag for case-insensitive matching

Recursive Search (searchInObject)​

const searchInObject = (obj: JSONObject, regex: RegExp): boolean => {
  for (const [key, value] of Object.entries(obj)) {
    // Skip internal properties
    if (key.startsWith('__')) continue;
    
    // Search in string values
    if (typeof value === 'string' && regex.test(value)) {
      return true;
    }
    
    // Search in number values converted to string
    if (typeof value === 'number' && regex.test(String(value))) {
      return true;
    }
    
    // Search in nested objects and arrays
    if (value !== null && typeof value === 'object') {
      if (Array.isArray(value)) {
        for (const item of value) {
          if (typeof item === 'object' && item !== null && searchInObject(item as JSONObject, regex)) {
            return true;
          } else if ((typeof item === 'string' && regex.test(item)) || 
                     (typeof item === 'number' && regex.test(String(item)))) {
            return true;
          }
        }
      } else if (searchInObject(value as JSONObject, regex)) {
        return true;
      }
    }
  }
  
  return false;
};

Deep search capabilities:

  • String matching - Direct regex test on string values
  • Number matching - Converts numbers to strings for searching
  • Nested objects - Recursively searches child objects
  • Array elements - Searches both primitive and object array elements
  • Skips internals - Ignores __ prefixed properties

Data Sorting (sortData)​

export const sortData = (
  data: JSONObject[], 
  key: string, 
  direction: 'asc' | 'desc'
): JSONObject[] => {
  return [...data].sort((a, b) => {
    const valueA = getValueByPath(a, key);
    const valueB = getValueByPath(b, key);
    
    // Handle undefined or null values
    if (valueA === undefined || valueA === null) return direction === 'asc' ? -1 : 1;
    if (valueB === undefined || valueB === null) return direction === 'asc' ? 1 : -1;
    
    // Compare based on type
    if (typeof valueA === 'string' && typeof valueB === 'string') {
      return direction === 'asc' 
        ? valueA.localeCompare(valueB)
        : valueB.localeCompare(valueA);
    }
    
    if (typeof valueA === 'number' && typeof valueB === 'number') {
      return direction === 'asc' ? valueA - valueB : valueB - valueA;
    }
    
    // Convert to string for mixed types
    const strA = String(valueA);
    const strB = String(valueB);
    
    return direction === 'asc' ? strA.localeCompare(strB) : strB.localeCompare(strA);
  });
};

Purpose: Sorts data by a specified field with proper type handling.

Sorting logic:

  1. Immutable - Creates a copy with [...data] before sorting
  2. Path support - Uses getValueByPath for nested field access
  3. Null handling - Places null/undefined values at the beginning (asc) or end (desc)
  4. Type-specific comparison:
    • Strings - Uses localeCompare() for proper locale-aware sorting
    • Numbers - Uses arithmetic comparison for numerical sorting
    • Mixed types - Converts to strings as fallback
  5. Direction support - Reverses comparison for descending order

Key Design Principles​

Immutability​

Functions like sortData create new arrays rather than modifying input data, preventing unintended side effects.

Type Safety​

All functions include proper TypeScript typing and runtime type checking to handle the dynamic nature of JSON data.

Error Resilience​

Functions gracefully handle edge cases like null values, invalid regex patterns, and missing object properties.

Performance Optimization​

  • Uses Set for deduplication in extractKeys
  • Implements early returns in search functions
  • Minimizes object traversal where possible

These utilities form the foundation for all data manipulation in the application, providing reliable and efficient processing of JSON data structures.