[[tooling/Software Development/Frameworks/Next.js]]
[[concepts/Data Augmentation Workflow|Data Workflows]]
[[organizations/Meta|Meta Platforms]]

> [!<Class-Definition>] <Heading in Callout>
> 
> <content>

> [!info] Integration Notice
> 
> This component integrates with the shared authentication service.

> [!warning] Breaking Changes
> 
> Version 2.0 introduces breaking changes to the API interface.

> [!tip] Performance Optimization
> 
> Use the `--no-cache` flag for development builds.

:::directive-name
content
:::

```toolingGallery small
- tag: [[AI-Toolkit]]
- [[tooling/AI-Toolkit/OpenAI]]
- [[tooling/AI-Toolkit/Anthropic]]

**Mermaid Diagrams**:
```markdown
```mermaid
graph TD
    A[Start] --> B[Process]
    B --> C[End]

## 1. Executive Summary
The Flavored Markdown Parser Service is a comprehensive content processing system designed specifically for the Augment-It platform's rich content ecosystem. It extends standard markdown with powerful features including bidirectional linking (wikilinks), styled callouts, interactive directives, and content collection integration. Built on the robust remark/rehype ecosystem, it provides semantic parsing, content validation, link resolution, and component rendering capabilities that enable a sophisticated knowledge management and content creation workflow.

## 2. Background & Motivation
### Problem Statement
The Augment-It platform requires advanced markdown processing capabilities that go beyond standard markdown to support knowledge management, content cross-referencing, and interactive component embedding across thousands of interconnected documents.

### Current Limitations
- **Standard Markdown Constraints**: Basic markdown lacks semantic linking and content organization features
- **Manual Cross-Referencing**: No automated way to link related content across collections
- **Static Content**: Limited ability to embed dynamic or interactive components
- **Inconsistent Styling**: No standardized way to create styled content blocks
- **Content Isolation**: Documents exist in isolation without semantic relationships

### Why This Solution
- **Knowledge Graph Integration**: Enables bidirectional linking and content discovery
- **Component Ecosystem**: Supports rich, interactive content through directives
- **Content Collections**: Seamless integration with organized content taxonomies
- **Extensible Architecture**: Plugin-based system for custom syntax extensions
- **Performance Optimized**: Efficient processing of large content repositories

## 3. Goals & Non-Goals
### Goals
1. **Extended Syntax Support**: Comprehensive parsing of wikilinks, callouts, and directives
2. **Link Resolution**: Automatic resolution and validation of internal content links
3. **Component Integration**: Seamless embedding of interactive components via directives
4. **Content Collections**: Deep integration with taxonomized content organization
5. **Performance**: Efficient processing of large content repositories
6. **Extensibility**: Plugin architecture for custom syntax extensions
7. **Error Handling**: Graceful handling of malformed syntax and missing references

### Non-Goals
1. **WYSIWYG Editing**: Focus on parsing, not visual editing interfaces
2. **Real-time Collaboration**: Batch processing focus, not collaborative editing
3. **Version Control**: Markdown processing only, not content versioning
4. **Content Management**: Parsing service, not full CMS functionality

## 4. Technical Design
### High-Level Architecture
```mermaid
graph TD
    A[Markdown Input] --> B[Flavored Markdown Parser]
    B --> C[Syntax Analyzer]
    C --> D[Wikilink Resolver]
    C --> E[Callout Processor]
    C --> F[Directive Handler]
    D --> G[Content Collections]
    E --> H[Styled Components]
    F --> I[Interactive Components]
    G --> J[Link Validation]
    H --> K[Rendered Output]
    I --> K
    J --> K
    
    L[Remark Plugins] --> C
    M[Rehype Plugins] --> K
    N[Component Registry] --> F

interface FlavoredMarkdownOptions {
  enableWikilinks?: boolean; // Default: true
  enableCallouts?: boolean; // Default: true
  enableDirectives?: boolean; // Default: true
  strictLinkValidation?: boolean; // Default: false
  baseUrl?: string; // For absolute URL generation
  contentCollections?: string[]; // Available collections
  componentRegistry?: ComponentRegistry;
  customSyntax?: CustomSyntaxPlugin[];
}

interface ParseResult {
  success: boolean;
  ast?: any; // Markdown AST
  html?: string; // Rendered HTML
  metadata: {
    wikilinks: WikilinkInfo[];
    callouts: CalloutInfo[];
    directives: DirectiveInfo[];
    errors: ParseError[];
    warnings: ParseWarning[];
    processingTime: number;
  };
}

interface WikilinkInfo {
  originalText: string;
  filePath: string;
  displayText?: string;
  resolved: boolean;
  resolvedUrl?: string;
  collection?: string;
  line: number;
  column: number;
}

interface CalloutInfo {
  type: 'info' | 'warning' | 'error' | 'success' | 'note' | 'tip';
  title?: string;
  content: string;
  line: number;
}

interface DirectiveInfo {
  type: 'leaf' | 'container';
  name: string;
  attributes: Record<string, any>;
  content?: string;
  component?: string;
  resolved: boolean;
  line: number;
}

// Main parsing functions
function parseFlavoredMarkdown(content: string, options?: FlavoredMarkdownOptions): Promise<ParseResult>;
function resolveWikilinks(content: string, collections: ContentCollection[]): Promise<WikilinkInfo[]>;
function validateLinks(content: string, options?: FlavoredMarkdownOptions): Promise<LinkValidationResult>;
function extractDirectives(content: string): DirectiveInfo[];
function renderToHtml(content: string, options?: FlavoredMarkdownOptions): Promise<string>;

// Based on existing implementations from AstroMarkdown.astro and remark plugins
class FlavoredMarkdownParser {
  private options: Required<FlavoredMarkdownOptions>;
  private remarkProcessor: any;
  private rehypeProcessor: any;
  private componentRegistry: ComponentRegistry;
  private contentCollections: Map<string, any[]>;
  
  constructor(options: FlavoredMarkdownOptions = {}) {
    this.options = {
      enableWikilinks: true,
      enableCallouts: true,
      enableDirectives: true,
      strictLinkValidation: false,
      baseUrl: '',
      contentCollections: [],
      componentRegistry: new ComponentRegistry(),
      customSyntax: [],
      ...options
    };
    
    this.initializeProcessors();
  }

  private initializeProcessors() {
    // Initialize remark processor with plugins
    this.remarkProcessor = remark()
      .use(remarkGfm) // GitHub Flavored Markdown
      .use(remarkFrontmatter) // YAML frontmatter
      .use(remarkDirective) // Directive support
      .use(this.remarkWikilinks.bind(this)) // Custom wikilink plugin
      .use(this.remarkCallouts.bind(this)) // Custom callout plugin
      .use(this.remarkDirectiveToComponent.bind(this)); // Custom directive plugin
    
    // Initialize rehype processor
    this.rehypeProcessor = rehype()
      .use(rehypeRaw) // Allow raw HTML
      .use(rehypeStringify); // Convert to HTML
  }

  // Wikilink processing plugin
  private remarkWikilinks() {
    return (tree: any) => {
      visit(tree, 'text', (node: any, index: number, parent: any) => {
        if (!this.options.enableWikilinks) return;
        
        const wikilinkRegex = /\[\[([^\]|]+)(?:\|([^\]]+))?\]\]/g;
        let match;
        const replacements = [];
        
        while ((match = wikilinkRegex.exec(node.value)) !== null) {
          const [fullMatch, filePath, displayText] = match;
          const resolvedLink = this.resolveWikilink(filePath, displayText);
          
          replacements.push({
            start: match.index,
            end: match.index + fullMatch.length,
            replacement: resolvedLink
          });
        }
        
        if (replacements.length > 0) {
          this.applyTextReplacements(node, parent, index, replacements);
        }
      });
    };
  }

  // Callout processing plugin
  private remarkCallouts() {
    return (tree: any) => {
      visit(tree, 'blockquote', (node: any) => {
        if (!this.options.enableCallouts) return;
        
        // Check if this is a callout blockquote
        const firstChild = node.children[0];
        if (firstChild && firstChild.type === 'paragraph') {
          const firstText = this.getTextContent(firstChild);
          const calloutMatch = firstText.match(/^\[!([^\]]+)\]\s*(.*)/);
          
          if (calloutMatch) {
            const [, type, title] = calloutMatch;
            this.transformToCallout(node, type.toLowerCase(), title);
          }
        }
      });
    };
  }

  // Directive processing plugin
  private remarkDirectiveToComponent() {
    return (tree: any) => {
      visit(tree, ['leafDirective', 'containerDirective'], (node: any) => {
        if (!this.options.enableDirectives) return;
        
        const directiveName = node.name;
        const component = this.componentRegistry.getComponent(directiveName);
        
        if (component) {
          // Transform directive to component call
          node.type = 'html';
          node.value = this.renderDirectiveAsHtml(node, component);
        } else {
          // Log warning for unknown directive
          console.warn(`Unknown directive: ${directiveName}`);
        }
      });
    };
  }

  // Wikilink resolution
  private resolveWikilink(filePath: string, displayText?: string): any {
    // Clean up the file path
    const cleanPath = filePath.trim();
    const linkText = displayText || cleanPath.split('/').pop() || cleanPath;
    
    // Try to resolve against content collections
    const resolvedUrl = this.findInContentCollections(cleanPath);
    
    if (resolvedUrl) {
      return {
        type: 'link',
        url: resolvedUrl,
        children: [{ type: 'text', value: linkText }]
      };
    } else {
      // Return broken link with warning styling
      return {
        type: 'html',
        value: `<span class="broken-link" title="Link not found: ${cleanPath}">${linkText}</span>`
      };
    }
  }

  // Content collection search
  private findInContentCollections(filePath: string): string | null {
    for (const [collectionName, items] of this.contentCollections.entries()) {
      for (const item of items) {
        if (item.id === filePath || item.slug === filePath) {
          return this.generateUrl(collectionName, item);
        }
      }
    }
    return null;
  }

  // Callout transformation
  private transformToCallout(node: any, type: string, title: string) {
    // Extract content after the title
    const content = this.extractCalloutContent(node);
    
    // Transform to custom callout HTML
    node.type = 'html';
    node.value = `
      <div class="callout callout-${type}">
        ${title ? `<div class="callout-title">
          <span class="callout-icon">${this.getCalloutIcon(type)}</span>
          <span class="callout-title-text">${title}</span>
        </div>` : ''}
        <div class="callout-content">
          ${content}
        </div>
      </div>
    `;
  }

  // Directive rendering
  private renderDirectiveAsHtml(node: any, component: ComponentInfo): string {
    const attributes = this.parseDirectiveAttributes(node.attributes || {});
    
    // Handle different directive types
    if (node.type === 'leafDirective') {
      return `<${component.tagName} ${this.attributesToString(attributes)} />`;
    } else if (node.type === 'containerDirective') {
      const content = this.getTextContent(node);
      return `<${component.tagName} ${this.attributesToString(attributes)}>${content}</${component.tagName}>`;
    }
    
    return '';
  }

  // Main parsing method
  public async parse(content: string): Promise<ParseResult> {
    const startTime = Date.now();
    const metadata = {
      wikilinks: [],
      callouts: [],
      directives: [],
      errors: [],
      warnings: [],
      processingTime: 0
    };
    
    try {
      // Process through remark pipeline
      const remarkResult = await this.remarkProcessor.process(content);
      
      // Extract metadata during processing
      this.extractMetadata(remarkResult, metadata);
      
      // Convert to HTML if needed
      const rehypeResult = await this.rehypeProcessor.process(remarkResult);
      
      metadata.processingTime = Date.now() - startTime;
      
      return {
        success: true,
        ast: remarkResult,
        html: String(rehypeResult),
        metadata
      };
    } catch (error) {
      metadata.errors.push({
        message: error instanceof Error ? error.message : 'Unknown parsing error',
        line: -1,
        column: -1,
        code: 'PARSE_ERROR',
        severity: 'error'
      });
      
      return {
        success: false,
        metadata
      };
    }
  }

  // Content collection integration
  public loadContentCollections(collections: Record<string, any[]>) {
    this.contentCollections = new Map(Object.entries(collections));
  }

  // Custom syntax plugin registration
  public registerCustomSyntax(plugin: CustomSyntaxPlugin) {
    this.options.customSyntax.push(plugin);
    this.reinitializeProcessors();
  }
}

// Supporting interfaces and classes
class ComponentRegistry {
  private components = new Map<string, ComponentInfo>();
  
  register(name: string, component: ComponentInfo) {
    this.components.set(name, component);
  }
  
  getComponent(name: string): ComponentInfo | null {
    return this.components.get(name) || null;
  }
}

interface ComponentInfo {
  tagName: string;
  attributes: Record<string, any>;
  requiredAuth?: boolean;
}

interface CustomSyntaxPlugin {
  name: string;
  type: 'remark' | 'rehype';
  plugin: any;
  options?: any;
}