Class: Mongo::GridIO

Inherits:
Object show all
Defined in:
lib/mongo/gridfs/grid_io.rb,
lib/mongo/gridfs/grid_io_fix.rb

Overview

GridIO objects represent files in the GridFS specification. This class manages the reading and writing of file chunks and metadata.

Constant Summary

DEFAULT_CHUNK_SIZE =
256 * 1024
DEFAULT_CONTENT_TYPE =
'binary/octet-stream'
PROTECTED_ATTRS =
[:files_id, :file_length, :client_md5, :server_md5]

Instance Attribute Summary (collapse)

Instance Method Summary (collapse)

Constructor Details

- (GridIO) initialize(files, chunks, filename, mode, opts = {})

Create a new GridIO object. Note that most users will not need to use this class directly; the Grid and GridFileSystem classes will instantiate this class

Parameters:

  • (Mongo::Collection) files

    a collection for storing file metadata.

  • (Mongo::Collection) chunks

    a collection for storing file chunks.

  • (String) filename

    the name of the file to open or write.

  • (String) mode

    ‘r’ or ‘w’ or reading or creating a file.

  • (Hash) opts (defaults to: {})

    a customizable set of options

Options Hash (opts):

  • (Hash) :query

    a query selector used when opening the file in ‘r’ mode.

  • (Hash) :query_opts

    any query options to be used when opening the file in ‘r’ mode.

  • (String) :fs_name

    the file system prefix.

  • (Integer) (262144)

    :chunk_size size of file chunks in bytes.

  • (Hash) :metadata — default: {}

    any additional data to store with the file.

  • (ObjectId) :_id — default: ObjectId

    a unique id for the file to be use in lieu of an automatically generated one.

  • (String) :content_type — default: 'binary/octet-stream'

    If no content type is specified, the content type will may be inferred from the filename extension if the mime-types gem can be loaded. Otherwise, the content type ‘binary/octet-stream’ will be used.

  • (Boolean) :safe — default: false

    When safe mode is enabled, the chunks sent to the server will be validated using an md5 hash. If validation fails, an exception will be raised.



57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
# File 'lib/mongo/gridfs/grid_io.rb', line 57

def initialize(files, chunks, filename, mode, opts={})
  @files        = files
  @chunks       = chunks
  @filename     = filename
  @mode         = mode
  @query        = opts.delete(:query) || {}
  @query_opts   = opts.delete(:query_opts) || {}
  @fs_name      = opts.delete(:fs_name) || Grid::DEFAULT_FS_NAME
  @safe         = opts.delete(:safe) || false
  @local_md5    = Digest::MD5.new if @safe
  @custom_attrs = {}

  case @mode
    when 'r' then init_read
    when 'w' then init_write(opts)
    else
      raise GridError, "Invalid file mode #{@mode}. Mode should be 'r' or 'w'."
  end
end

Instance Attribute Details

- (Object) chunk_size (readonly)

Returns the value of attribute chunk_size



34
35
36
# File 'lib/mongo/gridfs/grid_io.rb', line 34

def chunk_size
  @chunk_size
end

- (Object) client_md5 (readonly)

Returns the value of attribute client_md5



34
35
36
# File 'lib/mongo/gridfs/grid_io.rb', line 34

def client_md5
  @client_md5
end

- (Object) content_type (readonly)

Returns the value of attribute content_type



34
35
36
# File 'lib/mongo/gridfs/grid_io.rb', line 34

def content_type
  @content_type
end

- (Object) file_length (readonly)

Returns the value of attribute file_length



34
35
36
# File 'lib/mongo/gridfs/grid_io.rb', line 34

def file_length
  @file_length
end

- (Object) filename (readonly)

Returns the value of attribute filename



34
35
36
# File 'lib/mongo/gridfs/grid_io.rb', line 34

def filename
  @filename
end

- (Object) files_id (readonly)

Returns the value of attribute files_id



34
35
36
# File 'lib/mongo/gridfs/grid_io.rb', line 34

def files_id
  @files_id
end

- (Object) metadata (readonly)

Returns the value of attribute metadata



34
35
36
# File 'lib/mongo/gridfs/grid_io.rb', line 34

def 
  
end

- (Object) server_md5 (readonly)

Returns the value of attribute server_md5



34
35
36
# File 'lib/mongo/gridfs/grid_io.rb', line 34

def server_md5
  @server_md5
end

- (Object) upload_date (readonly)

Returns the value of attribute upload_date



34
35
36
# File 'lib/mongo/gridfs/grid_io.rb', line 34

def upload_date
  @upload_date
end

Instance Method Details

- (Object) [](key)



77
78
79
# File 'lib/mongo/gridfs/grid_io.rb', line 77

def [](key)
  @custom_attrs[key] || instance_variable_get("@#{key.to_s}")
end

- (Object) []=(key, value)



81
82
83
84
85
86
87
88
# File 'lib/mongo/gridfs/grid_io.rb', line 81

def []=(key, value)
  if PROTECTED_ATTRS.include?(key.to_sym)
    warn "Attempting to overwrite protected value."
    return nil
  else
    @custom_attrs[key] = value
  end
end

- (Object) check_existing_file



324
325
326
327
328
# File 'lib/mongo/gridfs/grid_io.rb', line 324

def check_existing_file
  if @files.find_one('_id' => @files_id)
    raise GridError, "Attempting to overwrite with Grid#put. You must delete the file first."
  end
end

- (BSON::ObjectId) close

Creates or updates the document from the files collection that stores the chunks’ metadata. The file becomes available only after this method has been called.

This method will be invoked automatically when on GridIO#open is passed a block. Otherwise, it must be called manually.

Returns:



184
185
186
187
188
189
190
191
192
193
# File 'lib/mongo/gridfs/grid_io.rb', line 184

def close
  if @mode[0] == ?w
    if @current_chunk['n'].zero? && @chunk_position.zero?
      warn "Warning: Storing a file with zero length."
    end
    @upload_date = Time.now.utc
    id = @files.insert(to_mongo_object)
  end
  id
end

- (Object) get_content_type

Determine the content type based on the filename.



362
363
364
365
366
367
368
# File 'lib/mongo/gridfs/grid_io.rb', line 362

def get_content_type
  if @filename
    if types = MIME::Types.type_for(@filename)
      types.first.simplified unless types.empty?
    end
  end
end

- (Object) get_md5

This fixes a comparson issue in JRuby 1.9



346
347
348
349
350
351
352
353
354
355
356
357
358
359
# File 'lib/mongo/gridfs/grid_io.rb', line 346

def get_md5
  md5_command            = BSON::OrderedHash.new
  md5_command['filemd5'] = @files_id
  md5_command['root']    = @fs_name
  @server_md5 = @files.db.command(md5_command)['md5']
  if @safe
    @client_md5 = @local_md5.hexdigest
    if @local_md5.to_s != @server_md5.to_s
      raise GridMD5Failure, "File on server failed MD5 check"
    end
  else
    @server_md5
  end
end

- (Object) init_read

Initialize the class for reading a file.

Raises:



290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
# File 'lib/mongo/gridfs/grid_io.rb', line 290

def init_read
  doc = @files.find(@query, @query_opts).next_document
  raise GridFileNotFound, "Could not open file matching #{@query.inspect} #{@query_opts.inspect}" unless doc

  @files_id     = doc['_id']
  @content_type = doc['contentType']
  @chunk_size   = doc['chunkSize']
  @upload_date  = doc['uploadDate']
  @aliases      = doc['aliases']
  @file_length  = doc['length']
       = doc['metadata']
  @md5          = doc['md5']
  @filename     = doc['filename']
  @custom_attrs = doc

  @current_chunk = get_chunk(0)
  @file_position = 0
end

- (Object) init_write(opts)

Initialize the class for writing a file.



310
311
312
313
314
315
316
317
318
319
320
321
322
# File 'lib/mongo/gridfs/grid_io.rb', line 310

def init_write(opts)
  @files_id      = opts.delete(:_id) || BSON::ObjectId.new
  @content_type  = opts.delete(:content_type) || (defined? MIME) && get_content_type || DEFAULT_CONTENT_TYPE
  @chunk_size    = opts.delete(:chunk_size) || DEFAULT_CHUNK_SIZE
        = opts.delete(:metadata) if opts[:metadata]
  @aliases       = opts.delete(:aliases) if opts[:aliases]
  @file_length   = 0
  opts.each {|k, v| self[k] = v}
  check_existing_file if @safe

  @current_chunk = create_chunk(0)
  @file_position = 0
end

- (Object) inspect



195
196
197
# File 'lib/mongo/gridfs/grid_io.rb', line 195

def inspect
  "#<GridIO _id: #{@files_id}>"
end

- (String) read(length = nil) Also known as: data

Read the data from the file. If a length if specified, will read from the current file position.

Parameters:

  • (Integer) length (defaults to: nil)

Returns:

  • (String)

    the data in the file



97
98
99
100
101
102
103
104
105
106
# File 'lib/mongo/gridfs/grid_io.rb', line 97

def read(length=nil)
  return '' if @file_length.zero?
  if length == 0
    return ''
  elsif length.nil? && @file_position.zero?
    read_all
  else
    read_length(length)
  end
end

- (Integer) seek(pos, whence = IO::SEEK_SET)

Position the file pointer at the provided location.

Parameters:

  • (Integer) pos

    the number of bytes to advance the file pointer. this can be a negative number.

  • (Integer) whence (defaults to: IO::SEEK_SET)

    one of IO::SEEK_CUR, IO::SEEK_END, or IO::SEEK_SET

Returns:

  • (Integer)

    the new file position

Raises:



148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
# File 'lib/mongo/gridfs/grid_io.rb', line 148

def seek(pos, whence=IO::SEEK_SET)
  raise GridError, "Seek is only allowed in read mode." unless @mode == 'r'
  target_pos = case whence
               when IO::SEEK_CUR
                 @file_position + pos
               when IO::SEEK_END
                 @file_length + pos
               when IO::SEEK_SET
                 pos
               end

  new_chunk_number = (target_pos / @chunk_size).to_i
  if new_chunk_number != @current_chunk['n']
    save_chunk(@current_chunk) if @mode[0] == ?w
    @current_chunk = get_chunk(new_chunk_number)
  end
  @file_position  = target_pos
  @chunk_position = @file_position % @chunk_size
  @file_position
end

- (Integer) tell

The current position of the file.

Returns:

  • (Integer)


172
173
174
# File 'lib/mongo/gridfs/grid_io.rb', line 172

def tell
  @file_position
end

- (Object) to_mongo_object



330
331
332
333
334
335
336
337
338
339
340
341
342
343
# File 'lib/mongo/gridfs/grid_io.rb', line 330

def to_mongo_object
  h                = BSON::OrderedHash.new
  h['_id']         = @files_id
  h['filename']    = @filename if @filename
  h['contentType'] = @content_type
  h['length']      = @current_chunk ? @current_chunk['n'] * @chunk_size + @chunk_position : 0
  h['chunkSize']   = @chunk_size
  h['uploadDate']  = @upload_date
  h['aliases']     = @aliases if @aliases
  h['metadata']    =  if 
  h['md5']         = get_md5
  h.merge!(@custom_attrs)
  h
end

- (Integer) write(io)

Write the given string (binary) data to the file.

Parameters:

  • (String) string

    the data to write

Returns:

  • (Integer)

    the number of bytes written.

Raises:



116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
# File 'lib/mongo/gridfs/grid_io.rb', line 116

def write(io)
  raise GridError, "file not opened for write" unless @mode[0] == ?w
  if io.is_a? String
    if @safe
      @local_md5.update(io)
    end
    write_string(io)
  else
    length = 0
    if @safe
      while(string = io.read(@chunk_size))
        @local_md5.update(string)
        length += write_string(string)
      end
    else
      while(string = io.read(@chunk_size))
        length += write_string(string)
      end
    end
    length
  end
end