Package org.elasticsearch.ingest

Class IngestDocument

java.lang.Object
org.elasticsearch.ingest.IngestDocument
public final class IngestDocument extends Object
Represents a single document being captured before indexing and holds the source and metadata (like id, type and index).

  • Field Details

  • Constructor Details

    • IngestDocument

      public IngestDocument(String index, String type, String id, String routing, Long version, VersionType versionType, Map<String,Object> source)

    • IngestDocument

      public IngestDocument(IngestDocument other)
      Copy constructor that creates a new IngestDocument which has exactly the same properties as the one provided as argument

    • IngestDocument

      public IngestDocument(Map<String,Object> sourceAndMetadata, Map<String,Object> ingestMetadata)
      Constructor needed for testing that allows to create a new IngestDocument given the provided elasticsearch metadata, source and ingest metadata. This is needed because the ingest metadata will be initialized with the current timestamp at init time, which makes equality comparisons impossible in tests.

  • Method Details

    • getFieldValue

      public <T> T getFieldValue(String path, Class<T> clazz)
      Returns the value contained in the document for the provided path
      Parameters:
      path - The path within the document in dot-notation
      clazz - The expected class of the field value
      Returns:
      the value for the provided path if existing, null otherwise
      Throws:
      IllegalArgumentException - if the path is null, empty, invalid, if the field doesn't exist or if the field that is found at the provided path is not of the expected type.

    • getFieldValue

      public <T> T getFieldValue(String path, Class<T> clazz, boolean ignoreMissing)
      Returns the value contained in the document for the provided path
      Parameters:
      path - The path within the document in dot-notation
      clazz - The expected class of the field value
      ignoreMissing - The flag to determine whether to throw an exception when `path` is not found in the document.
      Returns:
      the value for the provided path if existing, null otherwise.
      Throws:
      IllegalArgumentException - only if ignoreMissing is false and the path is null, empty, invalid, if the field doesn't exist or if the field that is found at the provided path is not of the expected type.

    • getFieldValue

      public <T> T getFieldValue(TemplateScript.Factory pathTemplate, Class<T> clazz)
      Returns the value contained in the document with the provided templated path
      Parameters:
      pathTemplate - The path within the document in dot-notation
      clazz - The expected class fo the field value
      Returns:
      the value fro the provided path if existing, null otherwise
      Throws:
      IllegalArgumentException - if the pathTemplate is null, empty, invalid, if the field doesn't exist, or if the field that is found at the provided path is not of the expected type.

    • getFieldValueAsBytes

      public byte[] getFieldValueAsBytes(String path)
      Returns the value contained in the document for the provided path as a byte array. If the path value is a string, a base64 decode operation will happen. If the path value is a byte array, it is just returned
      Parameters:
      path - The path within the document in dot-notation
      Returns:
      the byte array for the provided path if existing
      Throws:
      IllegalArgumentException - if the path is null, empty, invalid, if the field doesn't exist or if the field that is found at the provided path is not of the expected type.

    • getFieldValueAsBytes

      public byte[] getFieldValueAsBytes(String path, boolean ignoreMissing)
      Returns the value contained in the document for the provided path as a byte array. If the path value is a string, a base64 decode operation will happen. If the path value is a byte array, it is just returned
      Parameters:
      path - The path within the document in dot-notation
      ignoreMissing - The flag to determine whether to throw an exception when `path` is not found in the document.
      Returns:
      the byte array for the provided path if existing
      Throws:
      IllegalArgumentException - if the path is null, empty, invalid, if the field doesn't exist or if the field that is found at the provided path is not of the expected type.

    • hasField

      public boolean hasField(TemplateScript.Factory fieldPathTemplate)
      Checks whether the document contains a value for the provided templated path
      Parameters:
      fieldPathTemplate - the template for the path within the document in dot-notation
      Returns:
      true if the document contains a value for the field, false otherwise
      Throws:
      IllegalArgumentException - if the path is null, empty or invalid

    • hasField

      public boolean hasField(String path)
      Checks whether the document contains a value for the provided path
      Parameters:
      path - The path within the document in dot-notation
      Returns:
      true if the document contains a value for the field, false otherwise
      Throws:
      IllegalArgumentException - if the path is null, empty or invalid.

    • hasField

      public boolean hasField(String path, boolean failOutOfRange)
      Checks whether the document contains a value for the provided path
      Parameters:
      path - The path within the document in dot-notation
      failOutOfRange - Whether to throw an IllegalArgumentException if array is accessed outside of its range
      Returns:
      true if the document contains a value for the field, false otherwise
      Throws:
      IllegalArgumentException - if the path is null, empty or invalid.

    • removeField

      public void removeField(TemplateScript.Factory fieldPathTemplate)
      Removes the field identified by the provided path.
      Parameters:
      fieldPathTemplate - Resolves to the path with dot-notation within the document
      Throws:
      IllegalArgumentException - if the path is null, empty, invalid or if the field doesn't exist.

    • removeField

      public void removeField(String path)
      Removes the field identified by the provided path.
      Parameters:
      path - the path of the field to be removed
      Throws:
      IllegalArgumentException - if the path is null, empty, invalid or if the field doesn't exist.

    • appendFieldValue

      public void appendFieldValue(String path, Object value)
      Appends the provided value to the provided path in the document. Any non existing path element will be created. If the path identifies a list, the value will be appended to the existing list. If the path identifies a scalar, the scalar will be converted to a list and the provided value will be added to the newly created list. Supports multiple values too provided in forms of list, in that case all the values will be appended to the existing (or newly created) list.
      Parameters:
      path - The path within the document in dot-notation
      value - The value or values to append to the existing ones
      Throws:
      IllegalArgumentException - if the path is null, empty or invalid.

    • appendFieldValue

      public void appendFieldValue(String path, Object value, boolean allowDuplicates)
      Appends the provided value to the provided path in the document. Any non existing path element will be created. If the path identifies a list, the value will be appended to the existing list. If the path identifies a scalar, the scalar will be converted to a list and the provided value will be added to the newly created list. Supports multiple values too provided in forms of list, in that case all the values will be appended to the existing (or newly created) list.
      Parameters:
      path - The path within the document in dot-notation
      value - The value or values to append to the existing ones
      allowDuplicates - When false, any values that already exist in the field will not be added
      Throws:
      IllegalArgumentException - if the path is null, empty or invalid.

    • appendFieldValue

      public void appendFieldValue(TemplateScript.Factory fieldPathTemplate, ValueSource valueSource)
      Appends the provided value to the provided path in the document. Any non existing path element will be created. If the path identifies a list, the value will be appended to the existing list. If the path identifies a scalar, the scalar will be converted to a list and the provided value will be added to the newly created list. Supports multiple values too provided in forms of list, in that case all the values will be appended to the existing (or newly created) list.
      Parameters:
      fieldPathTemplate - Resolves to the path with dot-notation within the document
      valueSource - The value source that will produce the value or values to append to the existing ones
      Throws:
      IllegalArgumentException - if the path is null, empty or invalid.

    • appendFieldValue

      public void appendFieldValue(TemplateScript.Factory fieldPathTemplate, ValueSource valueSource, boolean allowDuplicates)
      Appends the provided value to the provided path in the document. Any non existing path element will be created. If the path identifies a list, the value will be appended to the existing list. If the path identifies a scalar, the scalar will be converted to a list and the provided value will be added to the newly created list. Supports multiple values too provided in forms of list, in that case all the values will be appended to the existing (or newly created) list.
      Parameters:
      fieldPathTemplate - Resolves to the path with dot-notation within the document
      valueSource - The value source that will produce the value or values to append to the existing ones
      allowDuplicates - When false, any values that already exist in the field will not be added
      Throws:
      IllegalArgumentException - if the path is null, empty or invalid.

    • setFieldValue

      public void setFieldValue(String path, Object value)
      Sets the provided value to the provided path in the document. Any non existing path element will be created. If the last item in the path is a list, the value will replace the existing list as a whole. Use appendFieldValue(String, Object) to append values to lists instead.
      Parameters:
      path - The path within the document in dot-notation
      value - The value to put in for the path key
      Throws:
      IllegalArgumentException - if the path is null, empty, invalid or if the value cannot be set to the item identified by the provided path.

    • setFieldValue

      public void setFieldValue(TemplateScript.Factory fieldPathTemplate, ValueSource valueSource)
      Sets the provided value to the provided path in the document. Any non existing path element will be created. If the last element is a list, the value will replace the existing list.
      Parameters:
      fieldPathTemplate - Resolves to the path with dot-notation within the document
      valueSource - The value source that will produce the value to put in for the path key
      Throws:
      IllegalArgumentException - if the path is null, empty, invalid or if the value cannot be set to the item identified by the provided path.

    • setFieldValue

      public void setFieldValue(TemplateScript.Factory fieldPathTemplate, ValueSource valueSource, boolean ignoreEmptyValue)
      Sets the provided value to the provided path in the document. Any non existing path element will be created. If the last element is a list, the value will replace the existing list.
      Parameters:
      fieldPathTemplate - Resolves to the path with dot-notation within the document
      valueSource - The value source that will produce the value to put in for the path key
      ignoreEmptyValue - The flag to determine whether to exit quietly when the value produced by TemplatedValue is null or empty
      Throws:
      IllegalArgumentException - if the path is null, empty, invalid or if the value cannot be set to the item identified by the provided path.

    • setFieldValue

      public void setFieldValue(TemplateScript.Factory fieldPathTemplate, Object value, boolean ignoreEmptyValue)
      Sets the provided value to the provided path in the document. Any non existing path element will be created. If the last element is a list, the value will replace the existing list.
      Parameters:
      fieldPathTemplate - Resolves to the path with dot-notation within the document
      value - The value to put in for the path key
      ignoreEmptyValue - The flag to determine whether to exit quietly when the value produced by TemplatedValue is null or empty
      Throws:
      IllegalArgumentException - if the path is null, empty, invalid or if the value cannot be set to the item identified by the provided path.

    • renderTemplate

      public String renderTemplate(TemplateScript.Factory template)

    • extractMetadata

      public Map<IngestDocument.Metadata,Object> extractMetadata()
      one time operation that extracts the metadata fields from the ingest document and returns them. Metadata fields that used to be accessible as ordinary top level fields will be removed as part of this call.

    • getMetadata

      public Map<IngestDocument.Metadata,Object> getMetadata()
      Does the same thing as extractMetadata() but does not mutate the map.

    • getIngestMetadata

      public Map<String,Object> getIngestMetadata()
      Returns the available ingest metadata fields, by default only timestamp, but it is possible to set additional ones. Use only for reading values, modify them instead using setFieldValue(String, Object) and removeField(String)

    • getSourceAndMetadata

      public Map<String,Object> getSourceAndMetadata()
      Returns the document including its metadata fields, unless extractMetadata() has been called, in which case the metadata fields will not be present anymore. Modify the document instead using setFieldValue(String, Object) and removeField(String)

    • deepCopyMap

      public static <K, V> Map<K,V> deepCopyMap(Map<K,V> source)

    • deepCopy

      public static Object deepCopy(Object value)

    • executePipeline

      public void executePipeline(Pipeline pipeline, BiConsumer<IngestDocument,Exception> handler)
      Executes the given pipeline with for this document unless the pipeline has already been executed for this document.
      Parameters:
      pipeline - the pipeline to execute
      handler - handles the result or failure

    • equals

      public boolean equals(Object obj)
      Overrides:
      equals in class Object

    • hashCode

      public int hashCode()
      Overrides:
      hashCode in class Object

    • toString

      public String toString()
      Overrides:
      toString in class Object