[Bug]: Missing Mermaid Flowcharts

[winterfx]

crawl4ai version

0.6.0

Expected Behavior

When I crawl a URL whose document contains some flow charts written in mermaid, the crawl result misses these flow charts.

Current Behavior

When crawling a URL, the document contains flow charts written in mermaid. However, the crawling process fails to capture or display these flow charts in the final result. Instead of including the mermaid diagrams as expected, the crawl output is missing them entirely.

Is this reproducible?

Yes

Inputs Causing the Bug

Steps to Reproduce

Code snippets

OS

macOS

Python version

3.11.7

Browser

No response

Browser version

No response

Error logs & Screenshots (if applicable)

No response

[aravindkarnam]

@winterfx Can you give one example URL for us to troubleshoot. Thanks.

[winterfx]

crwl https://deepwiki.com/bwmarrin/snowflake -o markdown

[aravindkarnam]

@winterfx Thanks for sharing this. This is a very interesting usecase. I can see in the html of the page it get's rendered as an svg. I guess it would be real awesome to have it back in mermaid syntax in generated markdown, especially for the interpretation by the LLMs.

I'll take this down as feature request and we'll plan it shortly.