org.apache.lucene.codecs.compressing

Class CompressingStoredFieldsFormat

java.lang.Object

org.apache.lucene.codecs.StoredFieldsFormat

org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat

public class CompressingStoredFieldsFormat
extends StoredFieldsFormat

A StoredFieldsFormat that compresses documents in chunks in order to improve the compression ratio.

For a chunk size of chunkSize bytes, this StoredFieldsFormat does not support documents larger than (2³¹ - chunkSize) bytes.

For optimal performance, you should use a MergePolicy that returns segments that have the biggest byte size first.

WARNING: This API is experimental and might change in incompatible ways in the next release.

Constructor Summary

Constructors

Constructor and Description
CompressingStoredFieldsFormat(String formatName, CompressionMode compressionMode, int chunkSize, int maxDocsPerChunk, int blockShift) Create a new CompressingStoredFieldsFormat with an empty segment suffix.
CompressingStoredFieldsFormat(String formatName, String segmentSuffix, CompressionMode compressionMode, int chunkSize, int maxDocsPerChunk, int blockShift) Create a new CompressingStoredFieldsFormat.

Method Summary

Modifier and Type	Method and Description
StoredFieldsReader	fieldsReader(Directory directory, SegmentInfo si, FieldInfos fn, IOContext context) Returns a StoredFieldsReader to load stored fields.
StoredFieldsWriter	fieldsWriter(Directory directory, SegmentInfo si, IOContext context) Returns a StoredFieldsWriter to write stored fields.
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

CompressingStoredFieldsFormat

public CompressingStoredFieldsFormat(String formatName,
                                     CompressionMode compressionMode,
                                     int chunkSize,
                                     int maxDocsPerChunk,
                                     int blockShift)

Create a new CompressingStoredFieldsFormat with an empty segment suffix.

See Also:

CompressingStoredFieldsFormat(String, String, CompressionMode, int, int, int)

CompressingStoredFieldsFormat

public CompressingStoredFieldsFormat(String formatName,
                                     String segmentSuffix,
                                     CompressionMode compressionMode,
                                     int chunkSize,
                                     int maxDocsPerChunk,
                                     int blockShift)

Create a new CompressingStoredFieldsFormat.

formatName is the name of the format. This name will be used in the file formats to perform codec header checks.

segmentSuffix is the segment suffix. This suffix is added to the result file name only if it's not the empty string.

The compressionMode parameter allows you to choose between compression algorithms that have various compression and decompression speeds so that you can pick the one that best fits your indexing and searching throughput. You should never instantiate two CompressingStoredFieldsFormats that have the same name but different CompressionModes.

chunkSize is the minimum byte size of a chunk of documents. A value of 1 can make sense if there is redundancy across fields. maxDocsPerChunk is an upperbound on how many docs may be stored in a single chunk. This is to bound the cpu costs for highly compressible data.

Higher values of chunkSize should improve the compression ratio but will require more memory at indexing time and might make document loading a little slower (depending on the size of your OS cache compared to the size of your index).

Parameters:

formatName - the name of the StoredFieldsFormat

compressionMode - the CompressionMode to use

chunkSize - the minimum number of bytes of a single chunk of stored documents

maxDocsPerChunk - the maximum number of documents in a single chunk

blockShift - the log in base 2 of number of chunks to store in an index block

See Also:

CompressionMode

Method Detail

fieldsReader

public StoredFieldsReader fieldsReader(Directory directory,
                                       SegmentInfo si,
                                       FieldInfos fn,
                                       IOContext context)
                                throws IOException

Description copied from class: StoredFieldsFormat

Returns a StoredFieldsReader to load stored fields.

Specified by:

fieldsReader in class StoredFieldsFormat

Throws:

IOException

fieldsWriter

public StoredFieldsWriter fieldsWriter(Directory directory,
                                       SegmentInfo si,
                                       IOContext context)
                                throws IOException

Description copied from class: StoredFieldsFormat

Returns a StoredFieldsWriter to write stored fields.

Specified by:

fieldsWriter in class StoredFieldsFormat

Throws:

IOException

toString

public String toString()

Overrides:

toString in class Object