Pipeline de embeddings para la base de datos vectorial Qdrant

Name: Pipeline de embeddings para la base de datos vectorial Qdrant
Rating: 4.5 (10 reviews)
Author: Ghaith Alsirawan

Intermedio

Este es unAIflujo de automatización del dominio deautomatización que contiene 13 nodos.Utiliza principalmente nodos como Ftp, ManualTrigger, SplitInBatches, EmbeddingsOpenAi, VectorStoreQdrant, combinando tecnología de inteligencia artificial para lograr automatización inteligente. Cargar JSON a Qdrant a través de FTP para el pipeline de embeddings de la base de datos vectorial

Requisitos previos

•Clave de API de OpenAI
•Información de conexión del servidor Qdrant

Nodos utilizados (13)

DocumentDefaultDataLoader

TextSplitterCharacterTextSplitter

Categoría

Inteligencia Artificial

Vista previa del flujo de trabajo

Visualización de las conexiones entre nodos, con soporte para zoom y panorámica

Almacén Vectorial Qdrant

Al hacer clic en 'Probar flujo de trabajo'

Embeddings OpenAI

Cargador de Datos Predeterminado

Separador de Texto por Caracteres

Listar todos los archivos

Iterar sobre un elemento

Descargando elemento

React Flow

Exportar flujo de trabajo

Copie la siguiente configuración JSON en n8n para importar y usar este flujo de trabajo

{
  "id": "YoUP55V241b9F2ze",
  "meta": {
    "instanceId": "35ec7a1e5284dd5dab4dac454bbb30405138d2784c99e56ef8887a4fa9cd1977",
    "templateCredsSetupCompleted": true
  },
  "name": "Qdrant Vector Database Embedding Pipeline",
  "tags": [],
  "nodes": [
    {
      "id": "934ffad4-c93e-40c1-b4fd-1c09b518a9c3",
      "name": "Almacén Vectorial Qdrant",
      "type": "@n8n/n8n-nodes-langchain.vectorStoreQdrant",
      "position": [
        460,
        -460
      ],
      "parameters": {
        "mode": "insert",
        "options": {},
        "qdrantCollection": {
          "__rl": true,
          "mode": "list",
          "value": "sv_lang_data",
          "cachedResultName": "sv_lang_data"
        },
        "embeddingBatchSize": 100
      },
      "credentials": {
        "qdrantApi": {
          "id": "vUb9tbEnXzu7uNUb",
          "name": "QdrantApi svenska"
        }
      },
      "typeVersion": 1.1
    },
    {
      "id": "4127d85d-45c9-4536-a15d-08af9dfdcfa8",
      "name": "Al hacer clic en 'Probar flujo de trabajo'",
      "type": "n8n-nodes-base.manualTrigger",
      "position": [
        -960,
        -460
      ],
      "parameters": {},
      "typeVersion": 1
    },
    {
      "id": "abb61b81-72e0-468e-855b-72402db828fc",
      "name": "Embeddings OpenAI",
      "type": "@n8n/n8n-nodes-langchain.embeddingsOpenAi",
      "position": [
        400,
        -240
      ],
      "parameters": {
        "options": {}
      },
      "credentials": {
        "openAiApi": {
          "id": "kftHaZgVKiB9BmKU",
          "name": "OpenAi account"
        }
      },
      "typeVersion": 1.2
    },
    {
      "id": "e9ae24be-6da9-4c04-b891-7e450f505e02",
      "name": "Cargador de Datos Predeterminado",
      "type": "@n8n/n8n-nodes-langchain.documentDefaultDataLoader",
      "position": [
        780,
        -180
      ],
      "parameters": {
        "options": {},
        "dataType": "binary"
      },
      "typeVersion": 1
    },
    {
      "id": "9aff896d-4edb-494c-b84f-ede4e47db1e3",
      "name": "Separador de Texto por Caracteres",
      "type": "@n8n/n8n-nodes-langchain.textSplitterCharacterTextSplitter",
      "position": [
        800,
        20
      ],
      "parameters": {
        "separator": "\"chunk_id\""
      },
      "typeVersion": 1
    },
    {
      "id": "a083a47e-a835-4323-86a8-a2eaed226aaa",
      "name": "Nota Adhesiva",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        -760,
        -680
      ],
      "parameters": {
        "color": 4,
        "width": 260,
        "height": 200,
        "content": "### Fetch JSON File List\n**Node:** FTP (all files)\n**Operation:** List\n**Path:** <file path>\n\nRecursively lists all .json files prepared for embedding."
      },
      "typeVersion": 1
    },
    {
      "id": "072ae9dc-c1cd-4ceb-954a-6b6b1b984e29",
      "name": "Nota Adhesiva1",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        -460,
        -660
      ],
      "parameters": {
        "color": 5,
        "height": 180,
        "content": "### Iterate Over Files\n**Node:** Loop Over Items\n\nBatches each file path individually for processing."
      },
      "typeVersion": 1
    },
    {
      "id": "08d852f2-f1de-42ce-b882-1dc1343ed967",
      "name": "Nota Adhesiva2",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        -160,
        -700
      ],
      "parameters": {
        "color": 4,
        "width": 420,
        "height": 220,
        "content": "### Download Each File\n**Node:** FTP (1 file download)\n\nDownloads the current file in binary form using:\n```\nPath = file_path/{{ $json.name }}\n```"
      },
      "typeVersion": 1
    },
    {
      "id": "905c3d74-2817-4aa3-865d-51e972cbbb5a",
      "name": "Nota Adhesiva3",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        920,
        -80
      ],
      "parameters": {
        "color": 3,
        "width": 320,
        "height": 400,
        "content": "### Parse JSON Document (Default Data Loader)\n**Node:** Default Data Loader\n**Loader Type**: binary\n- Converts JSON structure into a document format compatible with embedding.\n\n\n### Split into Smaller Chunks\n**Node:** Character Text Splitter\n**Split by:** \"chunk_id\" or custom logic based on chunk formatting\n\nOptional node if chunk size normalization is required before embedding."
      },
      "typeVersion": 1
    },
    {
      "id": "9fb8e5be-3ee1-42b4-a858-40bc6afcf457",
      "name": "Listar todos los archivos",
      "type": "n8n-nodes-base.ftp",
      "position": [
        -700,
        -460
      ],
      "parameters": {
        "path": "Oracle/AI/embedding/svenska",
        "operation": "list"
      },
      "credentials": {
        "ftp": {
          "id": "JufoKeNjsIgbCBWe",
          "name": "FTP account"
        }
      },
      "typeVersion": 1
    },
    {
      "id": "6f8d0390-5851-44ca-9712-0ae51f9a22ef",
      "name": "Iterar sobre un elemento",
      "type": "n8n-nodes-base.splitInBatches",
      "position": [
        -400,
        -460
      ],
      "parameters": {
        "options": {}
      },
      "typeVersion": 3
    },
    {
      "id": "1c89a4a9-ec68-4c48-b7bc-74f5b30d8ac2",
      "name": "Descargando elemento",
      "type": "n8n-nodes-base.ftp",
      "position": [
        -40,
        -440
      ],
      "parameters": {
        "path": "=Oracle/AI/embedding/svenska/{{ $json.name }}",
        "binaryPropertyName": "binary.data"
      },
      "credentials": {
        "ftp": {
          "id": "JufoKeNjsIgbCBWe",
          "name": "FTP account"
        }
      },
      "typeVersion": 1,
      "alwaysOutputData": true
    },
    {
      "id": "01ca4ee3-5f1c-4977-a7f9-88e46db580ad",
      "name": "Nota Adhesiva4",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        360,
        -960
      ],
      "parameters": {
        "width": 480,
        "height": 460,
        "content": "### Store in Vector DB\n**Node:** Qdrant Vector Store\n**Batch Size:** 100\n\n**Collection:** <collection_name>\nSends cleaned text chunks to OpenAI to get embeddings (1536 dim for text-embedding-ada-002)\n\n#### collection settings in Qdrant cluster\n```\nPUT /collections/{collection_name}\n{\n    \"vectors\": {\n      \"size\": 1536,\n      \"distance\": \"Cosine\"\n    }\n}\n```\nEmbed Chunks\n**Node:** Embeddings OpenAI\nPushes the embedded chunks (with metadata) into Qdrant for semantic retrieval."
      },
      "typeVersion": 1
    }
  ],
  "active": false,
  "pinData": {},
  "settings": {
    "executionOrder": "v1"
  },
  "versionId": "c71fca63-26e9-4795-9a00-942dab6d07ce",
  "connections": {
    "1c89a4a9-ec68-4c48-b7bc-74f5b30d8ac2": {
      "main": [
        [
          {
            "node": "934ffad4-c93e-40c1-b4fd-1c09b518a9c3",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "abb61b81-72e0-468e-855b-72402db828fc": {
      "ai_embedding": [
        [
          {
            "node": "934ffad4-c93e-40c1-b4fd-1c09b518a9c3",
            "type": "ai_embedding",
            "index": 0
          }
        ]
      ]
    },
    "9fb8e5be-3ee1-42b4-a858-40bc6afcf457": {
      "main": [
        [
          {
            "node": "6f8d0390-5851-44ca-9712-0ae51f9a22ef",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "6f8d0390-5851-44ca-9712-0ae51f9a22ef": {
      "main": [
        [],
        [
          {
            "node": "1c89a4a9-ec68-4c48-b7bc-74f5b30d8ac2",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "e9ae24be-6da9-4c04-b891-7e450f505e02": {
      "ai_document": [
        [
          {
            "node": "934ffad4-c93e-40c1-b4fd-1c09b518a9c3",
            "type": "ai_document",
            "index": 0
          }
        ]
      ]
    },
    "934ffad4-c93e-40c1-b4fd-1c09b518a9c3": {
      "main": [
        [
          {
            "node": "9fb8e5be-3ee1-42b4-a858-40bc6afcf457",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "9aff896d-4edb-494c-b84f-ede4e47db1e3": {
      "ai_textSplitter": [
        [
          {
            "node": "e9ae24be-6da9-4c04-b891-7e450f505e02",
            "type": "ai_textSplitter",
            "index": 0
          }
        ]
      ]
    },
    "4127d85d-45c9-4536-a15d-08af9dfdcfa8": {
      "main": [
        [
          {
            "node": "9fb8e5be-3ee1-42b4-a858-40bc6afcf457",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  }
}

Preguntas frecuentes

¿Cómo usar este flujo de trabajo?

Copie el código de configuración JSON de arriba, cree un nuevo flujo de trabajo en su instancia de n8n y seleccione "Importar desde JSON", pegue la configuración y luego modifique la configuración de credenciales según sea necesario.

¿En qué escenarios es adecuado este flujo de trabajo?

Intermedio - Inteligencia Artificial

¿Es de pago?

Este flujo de trabajo es completamente gratuito, puede importarlo y usarlo directamente. Sin embargo, tenga en cuenta que los servicios de terceros utilizados en el flujo de trabajo (como la API de OpenAI) pueden requerir un pago por su cuenta.