Add SEO files and real documentation content
- Add permissive robots.txt welcoming all crawlers including AI - Add dynamic sitemap.xml with all pages and tools - Add real documentation content (Getting Started, Installation, First Tool, Publishing, Providers) - Update docs route to use content from docs_content.py 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
parent
1ac3054aaa
commit
4318b6cec5
|
|
@ -0,0 +1,400 @@
|
|||
"""Documentation content for SmartTools web UI.
|
||||
|
||||
This module contains the actual documentation text that gets rendered
|
||||
on the /docs pages. Content is stored as markdown-ish HTML for simplicity.
|
||||
"""
|
||||
|
||||
DOCS = {
|
||||
"getting-started": {
|
||||
"title": "Getting Started",
|
||||
"description": "Learn how to install SmartTools and create your first AI-powered CLI tool",
|
||||
"content": """
|
||||
<p class="lead">SmartTools lets you build custom AI-powered CLI commands using simple YAML configuration.
|
||||
Create tools that work with any AI provider and compose them like Unix pipes.</p>
|
||||
|
||||
<h2 id="what-is-smarttools">What is SmartTools?</h2>
|
||||
<p>SmartTools is a lightweight personal tool builder that lets you:</p>
|
||||
<ul>
|
||||
<li><strong>Create custom CLI commands</strong> that call AI providers</li>
|
||||
<li><strong>Chain prompts with Python code</strong> for complex workflows</li>
|
||||
<li><strong>Use tools like Unix pipes</strong> - read from stdin, write to stdout</li>
|
||||
<li><strong>Share and discover tools</strong> through the registry</li>
|
||||
</ul>
|
||||
|
||||
<h2 id="quick-start">Quick Start</h2>
|
||||
<p>Get up and running in under a minute:</p>
|
||||
|
||||
<pre><code class="language-bash"># Install SmartTools
|
||||
pip install smarttools
|
||||
|
||||
# Create your first tool interactively
|
||||
smarttools create
|
||||
|
||||
# Or install a tool from the registry
|
||||
smarttools registry install official/summarize
|
||||
|
||||
# Use it!
|
||||
cat article.txt | summarize</code></pre>
|
||||
|
||||
<h2 id="how-it-works">How It Works</h2>
|
||||
<p>Each tool is a YAML file that defines:</p>
|
||||
<ol>
|
||||
<li><strong>Arguments</strong> - Custom flags your tool accepts</li>
|
||||
<li><strong>Steps</strong> - Prompts to send to AI or Python code to run</li>
|
||||
<li><strong>Output</strong> - How to format the final result</li>
|
||||
</ol>
|
||||
|
||||
<p>Here's a simple example:</p>
|
||||
<pre><code class="language-yaml">name: summarize
|
||||
version: "1.0.0"
|
||||
description: Summarize text using AI
|
||||
|
||||
arguments:
|
||||
- flag: --max-length
|
||||
variable: max_length
|
||||
default: "200"
|
||||
description: Maximum summary length in words
|
||||
|
||||
steps:
|
||||
- type: prompt
|
||||
provider: claude
|
||||
prompt: |
|
||||
Summarize the following text in {max_length} words or less:
|
||||
|
||||
{input}
|
||||
output_var: summary
|
||||
|
||||
output: "{summary}"</code></pre>
|
||||
|
||||
<h2 id="next-steps">Next Steps</h2>
|
||||
<ul>
|
||||
<li><a href="/docs/installation">Installation Guide</a> - Detailed setup instructions</li>
|
||||
<li><a href="/docs/first-tool">Your First Tool</a> - Step-by-step tutorial</li>
|
||||
<li><a href="/docs/providers">Providers</a> - Configure AI providers</li>
|
||||
<li><a href="/tools">Browse Tools</a> - Discover community tools</li>
|
||||
</ul>
|
||||
""",
|
||||
"headings": [
|
||||
("what-is-smarttools", "What is SmartTools?"),
|
||||
("quick-start", "Quick Start"),
|
||||
("how-it-works", "How It Works"),
|
||||
("next-steps", "Next Steps"),
|
||||
],
|
||||
},
|
||||
|
||||
"installation": {
|
||||
"title": "Installation",
|
||||
"description": "How to install SmartTools on your system",
|
||||
"parent": "getting-started",
|
||||
"content": """
|
||||
<p class="lead">SmartTools requires Python 3.8+ and works on Linux, macOS, and Windows.</p>
|
||||
|
||||
<h2 id="pip-install">Install with pip</h2>
|
||||
<p>The simplest way to install SmartTools:</p>
|
||||
<pre><code class="language-bash">pip install smarttools</code></pre>
|
||||
|
||||
<p>Or with pipx for isolated installation:</p>
|
||||
<pre><code class="language-bash">pipx install smarttools</code></pre>
|
||||
|
||||
<h2 id="verify">Verify Installation</h2>
|
||||
<pre><code class="language-bash">smarttools --version
|
||||
smarttools --help</code></pre>
|
||||
|
||||
<h2 id="configure-provider">Configure a Provider</h2>
|
||||
<p>SmartTools needs at least one AI provider configured. The easiest is Claude CLI:</p>
|
||||
|
||||
<pre><code class="language-bash"># Install Claude CLI (if you have an Anthropic API key)
|
||||
pip install claude-cli
|
||||
|
||||
# Or use OpenAI
|
||||
pip install openai
|
||||
|
||||
# Configure your provider
|
||||
smarttools config</code></pre>
|
||||
|
||||
<h2 id="wrapper-scripts">Wrapper Scripts Location</h2>
|
||||
<p>SmartTools installs wrapper scripts to <code>~/.local/bin/</code>. Make sure this is in your PATH:</p>
|
||||
<pre><code class="language-bash"># Add to ~/.bashrc or ~/.zshrc
|
||||
export PATH="$HOME/.local/bin:$PATH"</code></pre>
|
||||
|
||||
<h2 id="development-install">Development Installation</h2>
|
||||
<p>To contribute or modify SmartTools:</p>
|
||||
<pre><code class="language-bash">git clone https://gitea.brrd.tech/rob/SmartTools.git
|
||||
cd SmartTools
|
||||
pip install -e ".[dev]"</code></pre>
|
||||
""",
|
||||
"headings": [
|
||||
("pip-install", "Install with pip"),
|
||||
("verify", "Verify Installation"),
|
||||
("configure-provider", "Configure a Provider"),
|
||||
("wrapper-scripts", "Wrapper Scripts Location"),
|
||||
("development-install", "Development Installation"),
|
||||
],
|
||||
},
|
||||
|
||||
"first-tool": {
|
||||
"title": "Your First Tool",
|
||||
"description": "Create your first SmartTools command step by step",
|
||||
"parent": "getting-started",
|
||||
"content": """
|
||||
<p class="lead">Let's create a simple tool that explains code. You'll learn the basics of tool configuration.</p>
|
||||
|
||||
<h2 id="create-tool">Create the Tool</h2>
|
||||
<p>Run the interactive creator:</p>
|
||||
<pre><code class="language-bash">smarttools create</code></pre>
|
||||
|
||||
<p>Or create the file manually at <code>~/.smarttools/explain/config.yaml</code>:</p>
|
||||
<pre><code class="language-yaml">name: explain
|
||||
version: "1.0.0"
|
||||
description: Explain code or concepts in simple terms
|
||||
category: code
|
||||
|
||||
arguments:
|
||||
- flag: --level
|
||||
variable: level
|
||||
default: "beginner"
|
||||
description: "Explanation level: beginner, intermediate, or expert"
|
||||
|
||||
steps:
|
||||
- type: prompt
|
||||
provider: claude
|
||||
prompt: |
|
||||
Explain the following in simple terms suitable for a {level}:
|
||||
|
||||
{input}
|
||||
|
||||
Be concise but thorough. Use examples where helpful.
|
||||
output_var: explanation
|
||||
|
||||
output: "{explanation}"</code></pre>
|
||||
|
||||
<h2 id="test-it">Test Your Tool</h2>
|
||||
<pre><code class="language-bash"># Explain some code
|
||||
echo "def fib(n): return n if n < 2 else fib(n-1) + fib(n-2)" | explain
|
||||
|
||||
# Explain for an expert
|
||||
cat complex_algorithm.py | explain --level expert</code></pre>
|
||||
|
||||
<h2 id="understanding-config">Understanding the Config</h2>
|
||||
|
||||
<h3>Arguments</h3>
|
||||
<p>Each argument becomes a CLI flag. The <code>variable</code> name is used in templates:</p>
|
||||
<pre><code class="language-yaml">arguments:
|
||||
- flag: --level # CLI flag: --level beginner
|
||||
variable: level # Use as {level} in prompts
|
||||
default: "beginner" # Default if not specified</code></pre>
|
||||
|
||||
<h3>Steps</h3>
|
||||
<p>Steps run in order. Each step can be a prompt or Python code:</p>
|
||||
<pre><code class="language-yaml">steps:
|
||||
- type: prompt
|
||||
provider: claude # Which AI to use
|
||||
prompt: "..." # The prompt template
|
||||
output_var: result # Store response in {result}</code></pre>
|
||||
|
||||
<h3>Output</h3>
|
||||
<p>The output template formats the final result:</p>
|
||||
<pre><code class="language-yaml">output: "{explanation}" # Print the explanation variable</code></pre>
|
||||
|
||||
<h2 id="next">Next Steps</h2>
|
||||
<ul>
|
||||
<li><a href="/docs/publishing">Publish your tool</a> to the registry</li>
|
||||
<li>Learn about <a href="/docs/providers">different providers</a></li>
|
||||
<li>See <a href="/tools">example tools</a> for inspiration</li>
|
||||
</ul>
|
||||
""",
|
||||
"headings": [
|
||||
("create-tool", "Create the Tool"),
|
||||
("test-it", "Test Your Tool"),
|
||||
("understanding-config", "Understanding the Config"),
|
||||
("next", "Next Steps"),
|
||||
],
|
||||
},
|
||||
|
||||
"publishing": {
|
||||
"title": "Publishing Tools",
|
||||
"description": "Share your tools with the SmartTools community",
|
||||
"content": """
|
||||
<p class="lead">Share your tools with the community by publishing to the SmartTools Registry.</p>
|
||||
|
||||
<h2 id="before-publishing">Before Publishing</h2>
|
||||
<p>Make sure your tool has:</p>
|
||||
<ul>
|
||||
<li>A descriptive <code>name</code> and <code>description</code></li>
|
||||
<li>A proper <code>version</code> (semver format: 1.0.0)</li>
|
||||
<li>A <code>README.md</code> file with usage examples</li>
|
||||
<li>Tested and working functionality</li>
|
||||
</ul>
|
||||
|
||||
<h2 id="create-account">Create an Account</h2>
|
||||
<p>Register at <a href="/register">the registry</a> to get your publisher namespace.</p>
|
||||
|
||||
<h2 id="get-token">Get an API Token</h2>
|
||||
<ol>
|
||||
<li>Go to your <a href="/dashboard/tokens">Dashboard → Tokens</a></li>
|
||||
<li>Click "Create New Token"</li>
|
||||
<li>Copy the token (shown only once!)</li>
|
||||
</ol>
|
||||
|
||||
<h2 id="publish">Publish Your Tool</h2>
|
||||
<pre><code class="language-bash"># Navigate to your tool directory
|
||||
cd ~/.smarttools/my-tool/
|
||||
|
||||
# First time: enter your token when prompted
|
||||
smarttools registry publish
|
||||
|
||||
# Dry run to validate without publishing
|
||||
smarttools registry publish --dry-run</code></pre>
|
||||
|
||||
<h2 id="versioning">Versioning</h2>
|
||||
<p>Published versions are <strong>immutable</strong>. To update a tool:</p>
|
||||
<ol>
|
||||
<li>Make your changes</li>
|
||||
<li>Bump the version in <code>config.yaml</code></li>
|
||||
<li>Run <code>smarttools registry publish</code></li>
|
||||
</ol>
|
||||
|
||||
<h2 id="best-practices">Best Practices</h2>
|
||||
<ul>
|
||||
<li><strong>Clear names</strong> - Use descriptive, lowercase names with hyphens</li>
|
||||
<li><strong>Good descriptions</strong> - Explain what the tool does in one sentence</li>
|
||||
<li><strong>Document arguments</strong> - Describe each flag in the help text</li>
|
||||
<li><strong>Include examples</strong> - Show real usage in your README</li>
|
||||
<li><strong>Choose the right category</strong> - Help users discover your tool</li>
|
||||
</ul>
|
||||
""",
|
||||
"headings": [
|
||||
("before-publishing", "Before Publishing"),
|
||||
("create-account", "Create an Account"),
|
||||
("get-token", "Get an API Token"),
|
||||
("publish", "Publish Your Tool"),
|
||||
("versioning", "Versioning"),
|
||||
("best-practices", "Best Practices"),
|
||||
],
|
||||
},
|
||||
|
||||
"providers": {
|
||||
"title": "AI Providers",
|
||||
"description": "Configure different AI providers for your tools",
|
||||
"content": """
|
||||
<p class="lead">SmartTools works with any AI provider that has a CLI interface. Configure providers in
|
||||
<code>~/.smarttools/providers.yaml</code>.</p>
|
||||
|
||||
<h2 id="provider-config">Provider Configuration</h2>
|
||||
<p>Create or edit <code>~/.smarttools/providers.yaml</code>:</p>
|
||||
|
||||
<pre><code class="language-yaml">providers:
|
||||
- name: claude
|
||||
command: "claude -p"
|
||||
|
||||
- name: openai
|
||||
command: "openai-cli"
|
||||
|
||||
- name: ollama
|
||||
command: "ollama run llama2"
|
||||
|
||||
- name: mock
|
||||
command: "echo '[MOCK RESPONSE]'"</code></pre>
|
||||
|
||||
<h2 id="using-providers">Using Providers in Tools</h2>
|
||||
<p>Specify the provider in your step:</p>
|
||||
<pre><code class="language-yaml">steps:
|
||||
- type: prompt
|
||||
provider: claude # Uses the "claude" provider from config
|
||||
prompt: "..."
|
||||
output_var: response</code></pre>
|
||||
|
||||
<h2 id="popular-providers">Popular Providers</h2>
|
||||
|
||||
<h3>Claude (Anthropic)</h3>
|
||||
<pre><code class="language-bash"># Install
|
||||
pip install claude-cli
|
||||
|
||||
# Configure with your API key
|
||||
export ANTHROPIC_API_KEY="sk-ant-..."</code></pre>
|
||||
|
||||
<pre><code class="language-yaml"># providers.yaml
|
||||
providers:
|
||||
- name: claude
|
||||
command: "claude -p"</code></pre>
|
||||
|
||||
<h3>OpenAI</h3>
|
||||
<pre><code class="language-bash"># Install
|
||||
pip install openai-cli
|
||||
|
||||
# Configure
|
||||
export OPENAI_API_KEY="sk-..."</code></pre>
|
||||
|
||||
<h3>Ollama (Local)</h3>
|
||||
<pre><code class="language-bash"># Install Ollama from ollama.ai
|
||||
# Pull a model
|
||||
ollama pull llama2</code></pre>
|
||||
|
||||
<pre><code class="language-yaml"># providers.yaml
|
||||
providers:
|
||||
- name: ollama
|
||||
command: "ollama run llama2"</code></pre>
|
||||
|
||||
<h2 id="testing">Testing with Mock Provider</h2>
|
||||
<p>Use the mock provider to test tools without API calls:</p>
|
||||
<pre><code class="language-yaml">providers:
|
||||
- name: mock
|
||||
command: "echo 'This is a mock response for testing'"</code></pre>
|
||||
|
||||
<h2 id="provider-selection">Choosing a Provider</h2>
|
||||
<table class="table">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Provider</th>
|
||||
<th>Best For</th>
|
||||
<th>Cost</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>Claude</td>
|
||||
<td>Complex reasoning, long context</td>
|
||||
<td>Pay per token</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>OpenAI</td>
|
||||
<td>General purpose, fast</td>
|
||||
<td>Pay per token</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Ollama</td>
|
||||
<td>Privacy, offline use</td>
|
||||
<td>Free (local)</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
""",
|
||||
"headings": [
|
||||
("provider-config", "Provider Configuration"),
|
||||
("using-providers", "Using Providers in Tools"),
|
||||
("popular-providers", "Popular Providers"),
|
||||
("testing", "Testing with Mock Provider"),
|
||||
("provider-selection", "Choosing a Provider"),
|
||||
],
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def get_doc(path: str) -> dict:
|
||||
"""Get documentation content by path."""
|
||||
# Normalize path
|
||||
path = path.strip("/").replace("docs/", "") or "getting-started"
|
||||
return DOCS.get(path, None)
|
||||
|
||||
|
||||
def get_toc():
|
||||
"""Get table of contents structure."""
|
||||
from types import SimpleNamespace
|
||||
return [
|
||||
SimpleNamespace(slug="getting-started", title="Getting Started", children=[
|
||||
SimpleNamespace(slug="installation", title="Installation"),
|
||||
SimpleNamespace(slug="first-tool", title="Your First Tool"),
|
||||
]),
|
||||
SimpleNamespace(slug="publishing", title="Publishing", children=[]),
|
||||
SimpleNamespace(slug="providers", title="Providers", children=[]),
|
||||
]
|
||||
|
|
@ -455,22 +455,30 @@ def dashboard_settings():
|
|||
@web_bp.route("/docs", defaults={"path": ""}, endpoint="docs")
|
||||
@web_bp.route("/docs/<path:path>", endpoint="docs")
|
||||
def docs(path: str):
|
||||
toc = [
|
||||
SimpleNamespace(slug="getting-started", title="Getting Started", children=[
|
||||
SimpleNamespace(slug="installation", title="Installation"),
|
||||
SimpleNamespace(slug="first-tool", title="Your First Tool"),
|
||||
]),
|
||||
SimpleNamespace(slug="publishing", title="Publishing", children=[]),
|
||||
SimpleNamespace(slug="providers", title="Providers", children=[]),
|
||||
]
|
||||
from .docs_content import get_doc, get_toc
|
||||
|
||||
toc = get_toc()
|
||||
current = path or "getting-started"
|
||||
|
||||
# Get content from docs_content module
|
||||
doc = get_doc(current)
|
||||
if doc:
|
||||
page = SimpleNamespace(
|
||||
title=doc["title"],
|
||||
description=doc["description"],
|
||||
content_html=Markup(doc["content"]),
|
||||
headings=doc.get("headings", []),
|
||||
parent=doc.get("parent"),
|
||||
)
|
||||
else:
|
||||
page = SimpleNamespace(
|
||||
title=_title_case(current),
|
||||
description="SmartTools documentation",
|
||||
content_html=f"<p>Documentation for <strong>{escape(current)}</strong> is coming soon.</p>",
|
||||
content_html=Markup(f"<p>Documentation for <strong>{escape(current)}</strong> is coming soon.</p>"),
|
||||
headings=[],
|
||||
parent=None,
|
||||
)
|
||||
|
||||
show_ads = current_app.config.get("SHOW_ADS", False)
|
||||
return render_template(
|
||||
"pages/docs.html",
|
||||
|
|
@ -538,3 +546,92 @@ def forgot_password():
|
|||
title="Reset Password",
|
||||
body="Password resets are not available yet. Please contact support if needed.",
|
||||
)
|
||||
|
||||
|
||||
@web_bp.route("/robots.txt", endpoint="robots")
|
||||
def robots():
|
||||
"""Serve robots.txt - we welcome all crawlers including AI."""
|
||||
from flask import send_from_directory
|
||||
return send_from_directory(
|
||||
web_bp.static_folder,
|
||||
"robots.txt",
|
||||
mimetype="text/plain"
|
||||
)
|
||||
|
||||
|
||||
@web_bp.route("/sitemap.xml", endpoint="sitemap")
|
||||
def sitemap():
|
||||
"""Generate dynamic sitemap.xml with all public pages and tools."""
|
||||
from flask import Response
|
||||
from datetime import datetime
|
||||
|
||||
conn = connect_db()
|
||||
try:
|
||||
tools = query_all(
|
||||
conn,
|
||||
"""
|
||||
WITH latest AS (
|
||||
SELECT owner, name, MAX(id) AS max_id
|
||||
FROM tools
|
||||
WHERE version NOT LIKE '%-%'
|
||||
GROUP BY owner, name
|
||||
)
|
||||
SELECT t.owner, t.name, t.published_at
|
||||
FROM tools t
|
||||
JOIN latest ON t.owner = latest.owner AND t.name = latest.name AND t.id = latest.max_id
|
||||
ORDER BY t.published_at DESC
|
||||
""",
|
||||
[],
|
||||
)
|
||||
tools = [dict(row) for row in tools]
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
base_url = request.url_root.rstrip("/")
|
||||
today = datetime.utcnow().strftime("%Y-%m-%d")
|
||||
|
||||
# Static pages
|
||||
static_pages = [
|
||||
("/", "1.0", "daily"),
|
||||
("/tools", "0.9", "daily"),
|
||||
("/docs", "0.8", "weekly"),
|
||||
("/docs/getting-started", "0.8", "weekly"),
|
||||
("/docs/installation", "0.7", "weekly"),
|
||||
("/docs/first-tool", "0.7", "weekly"),
|
||||
("/docs/publishing", "0.7", "weekly"),
|
||||
("/docs/providers", "0.7", "weekly"),
|
||||
("/tutorials", "0.7", "weekly"),
|
||||
("/about", "0.5", "monthly"),
|
||||
("/community", "0.5", "weekly"),
|
||||
("/donate", "0.4", "monthly"),
|
||||
("/privacy", "0.3", "monthly"),
|
||||
("/terms", "0.3", "monthly"),
|
||||
]
|
||||
|
||||
xml_parts = ['<?xml version="1.0" encoding="UTF-8"?>']
|
||||
xml_parts.append('<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">')
|
||||
|
||||
# Add static pages
|
||||
for path, priority, freq in static_pages:
|
||||
xml_parts.append(f""" <url>
|
||||
<loc>{base_url}{path}</loc>
|
||||
<lastmod>{today}</lastmod>
|
||||
<changefreq>{freq}</changefreq>
|
||||
<priority>{priority}</priority>
|
||||
</url>""")
|
||||
|
||||
# Add tool pages
|
||||
for tool in tools:
|
||||
pub_date = tool.get("published_at", today)
|
||||
if pub_date and "T" in str(pub_date):
|
||||
pub_date = str(pub_date).split("T")[0]
|
||||
xml_parts.append(f""" <url>
|
||||
<loc>{base_url}/tools/{tool['owner']}/{tool['name']}</loc>
|
||||
<lastmod>{pub_date or today}</lastmod>
|
||||
<changefreq>weekly</changefreq>
|
||||
<priority>0.6</priority>
|
||||
</url>""")
|
||||
|
||||
xml_parts.append("</urlset>")
|
||||
|
||||
return Response("\n".join(xml_parts), mimetype="application/xml")
|
||||
|
|
|
|||
|
|
@ -0,0 +1,47 @@
|
|||
# SmartTools Registry - robots.txt
|
||||
# We welcome all crawlers! Index everything to help users find us
|
||||
# and enable AI assistants to help solve SmartTools issues.
|
||||
|
||||
User-agent: *
|
||||
Allow: /
|
||||
|
||||
# Encourage full indexing of documentation and tutorials
|
||||
Allow: /docs/
|
||||
Allow: /tutorials/
|
||||
Allow: /tools/
|
||||
Allow: /categories/
|
||||
|
||||
# Allow AI crawlers explicitly (we want AI to learn about SmartTools)
|
||||
User-agent: GPTBot
|
||||
Allow: /
|
||||
|
||||
User-agent: ChatGPT-User
|
||||
Allow: /
|
||||
|
||||
User-agent: Claude-Web
|
||||
Allow: /
|
||||
|
||||
User-agent: Anthropic-AI
|
||||
Allow: /
|
||||
|
||||
User-agent: Google-Extended
|
||||
Allow: /
|
||||
|
||||
User-agent: CCBot
|
||||
Allow: /
|
||||
|
||||
User-agent: PerplexityBot
|
||||
Allow: /
|
||||
|
||||
# Only block private user areas (not useful to index)
|
||||
Disallow: /dashboard/
|
||||
Disallow: /api/v1/me/
|
||||
Disallow: /api/v1/tokens
|
||||
|
||||
# Allow API docs to be indexed (helpful for developers)
|
||||
Allow: /api/
|
||||
|
||||
# Sitemap location
|
||||
Sitemap: https://registry.smarttools.dev/sitemap.xml
|
||||
|
||||
# Crawl-delay: none - crawl as fast as you want!
|
||||
Loading…
Reference in New Issue