ScriptableIO
From MozillaWiki
These are some notes for adding better IO support for use in JS.
See bug 380813 for details.
The plan is to create a new object 'IO' which may be used to retrieve IO objects, currently files and streams, but other types could be added as well. This object is a singleton so there is only one of them and methods of this object are used to create new files and streams. The interface used for the IO object is described further down.
Examples:
// get the file stuff.txt from the Home directory.
// 'Home' is a key used in the directory service and 'stuff.txt' is a filename.
var file1 = IO.newFile("Home", "stuff.txt");
// get a direct path
var file2 = IO.newFileWithPath("/usr/local/myfile.txt");
// write a UTF-8 string to the file
var stream = IO.newOutputStream(file1, "text");
stream.writeString("This is a file\n");
stream.close();
The default character set for text input/output is UTF-8. This can be changed with an extra flag when creating the stream. Ideally, newInputStream and newOutputStream could take optional arguments, but I don't think idl supports this so this requires either a separate function or a C++ implementation and direct use of the JS API (as for example, XMLHttpRequest.open does)
The IO object has the following interface:
interface nsIScriptableIO : nsISupports
{
/**
* Retrieves a reference to a file or directory on disk, which may or may
* not exist. If the file exists, it may be opened for reading by passing
* the file as the base to newInputStream. If it doesn't exist, it may be
* created by opening an output stream and writing to it.
*
* Files are identified by the filename argument. The file is found relative
* to a well known directory identified by the location argument. This
* location is a string key which identifies common directories typically
* found on a system. For instance, using the location key 'Desk' will
* retrieve files in the desktop folder, and the location key 'TmpD' will
* retrieve files in the system's temporary directory.
*
* A complete list of location keys may be found at
* http://developer.mozilla.org/en/docs/IO_Guide/Directory_Keys
*
* The filename is always a file within the directory identified by the
* location key and this does not include a path. To retrieve
* subdirectories, retrieve a file and then use the file's append method
* to navigate into further subdirectories. This allows platform independent
* paths to be constructed.
*
* The filename may be a null string to retrieve a reference to the
* location directory itself.
*
* @param location location key of well-known directory
* @param filename filename to locate within this directory, may be null
* @param a file object
* @throws NS_ERROR_INVALID_ARG when aLocation is null
*/
nsIFile newFile(in AString aLocation, in AString aFileName);
/**
* Retrieves a reference to a file given a absolute file path.
*
* Use this method only when absolutely necessary. In most cases, newFile
* should be used instead, as file paths are not portable across different
* platforms and systems so this method should be avoided if possible.
* Instead, the newFile method should be used and a path constructed from
* it.
*
* The filepath should be an absolute path or the value of the
* persistentDescriptor of a file.
*
* @param filepath path to the file
* @param a file object
* @throws NS_ERROR_INVALID_ARG when aFilePath is null
*/
nsIFile newFileWithPath(in AString aFilePath);
/**
* Creates a URI object which implements nsIURI. The url argument may either
* be a string or a file.
*
* @param url the url to create
* @returns a new nsIURI object
* @throws NS_ERROR_INVALID_ARG when aUri is null
*/
nsIURI newURI(in nsIVariant aUri);
/**
* Retrieves a stream which may be read from.
*
* The base argument may be one of a number of different types of objects
* which may be read from:
* nsIFile - an object returned from the newFile or newFileWithPath
* methods, or any object which implements the nsIFile
* interface.
* nsITransport - a transport object such as a socket.
* nsIInputStream - a stream returned by a previous call to this method or
* any other object which implements the nsIInputStream
* interface.
* string - a string
*
* The mode may be any number of space separated strings which control
* the manner is which the stream is created. If no strings apply, use
* a null string. Possible values are:
* text - read unicode converted text. The default character set is UTF-8.
* To read text in a different character set, call
* newInputStreamWithArgs instead and supply the desired character
* set as a argument. The returned stream implements both the
* nsIConverterInputStream and nsIUnicodeLineInputStream
* interfaces.
* buffered - a stream which uses a buffer to hold a block of the next
* part of the data to read. This mode would normally be used
* as a wrapper for other streams. The size of the buffer
* defaults to 1024 bytes, however the size may be changed by
* supplying a value to the newInputStreamWithArgs method.
* If the text mode is used, the stream is always buffered.
* block - when reading from a transport such as a socket, an attempt to
* read from the stream while there is no data available will wait
* until data is available before returning. Without this mode,
* the stream will throw an exception if there is no data
* available.
* deleteonclose - the file is automatically deleted when the stream is
* closed. This might be used for temporary files.
* closeoneof - the file is automatically closed when the end of the file
* is reached.
* reopenonrewind - used in conjuction with the seek mode, the file will
* be reopened when a seek to the beginning of the file
* is done.
* multi - a stream which is used to concatenate the input from multiple
* streams together as if it was one long continuous stream. The
* returned stream implements the nsIMultiplexInputStream
* interface. This mode may only be used if the text or buffered
* modes are not used.
*
* If the mode is a null string, then no special type of reading is
* performed. In this case, the data from the stream is not interpreted in
* any way.
*
* @param base the base object to read from
* @param mode flags controlling the reading
* @returns a new input stream
* @throws NS_ERROR_INVALID_ARG when aBase is null
*/
nsISupports newInputStream(in nsIVariant aBase, in AString aMode);
/**
* Retrieves a stream which may be written to.
*
* The base argument may be one of a number of different types of objects
* which may be written to:
* nsIFile - an object returned from the newFile or newFileWithPath
* methods, or any object which implements the nsIFile
* interface.
* nsITransport - a transport object such as a socket.
* nsIOutputStream - a stream returned by a previous call to this method
* or any other object which implements the
* nsIOutputStream interface.
* string - a string
*
* The mode may be any number of space separated strings which control
* the manner is which the stream is created. If no strings apply, use
* a null string. Possible values are:
* text - write unicode converted text. The default character set is
* UTF-8. To write text in a different character set, call
* newOutputStreamWithArgs instead and supply the desired character
* set as a argument. The returned stream implements the
* nsIConverterOutputStream interface.
* buffered - a stream which buffers the data being written, which would
* normally be used as a wrapper for other streams. The size of
* the buffer defaults to 1024 bytes, however the size may be
* changed by supplying it a value to the
* newOutputStreamWithArgs method. If the text mode is used,
* the stream is always buffered.
* append - when writing to files, append to the end of the file instead
* of overwriting. If used in conjuction with the create mode, an
* existing file may be opened for appending, or a new file
* created.
* nocreate - when writing to files, and the file does not yet exist,
* don't create a new file. If this mode is not used, a
* new file will be created if it doesn't exist.
* notruncate - when writing to an existing file, overwrite the existing
* content. If this mode is not used, the file will be
* truncated to 0 length.
* syncsave - if used, then writing methods do not return until the
* data is properly saved.
* safe - when writing a file, writing is first performed to a temporary
* file and then when the stream's finish method is called, the
* temporary file is copied over to the real location. If an error
* occured at any time during writing, the temporary file is
* deleted and the original file is unaffected. This is useful when
* overwriting data that you want to ensure does not get lost.
* block - when writing to a transport such as a socket, an attempt
* to write to the stream will not return until all of the
* data has been written. This may cause a delay if the
* socket's underlying buffer is full. If this mode is not used,
* then an exception will be thrown if the buffer is full.
*
* If the mode is a null string, then no special type of writing is
* performed. In this case, the data being written to the stream is not
* interpreted in any way.
*
* @param base the base object to write to
* @param mode flags controlling the writing
* @returns a new output stream
* @throws NS_ERROR_INVALID_ARG when aBase is null
*/
nsISupports newOutputStream(in nsIVariant aBase, in AString aMode);
/**
* A variation of newInputStream which allows for additional arguments.
* The base and mode arguments work identically as to newInputStream.
*
* @param base the base object to read from
* @param mode flags controlling the reading
* @param bufferSize the size of buffer to use for buffered streams
* @param charset the character set to use when parsing text streams
* @param extra the replacement character for unknown characters
* @returns a new input stream
* @throws NS_ERROR_INVALID_ARG when aBase is null
*/
nsISupports newInputStreamWithArgs(in nsIVariant aBase,
in AString aMode,
in unsigned long aBufferSize,
in AString aCharSet,
in AString aExtra);
/**
* A variation of newOutputStream which allows for additional arguments.
* The base and mode arguments work identically as to newOutputStream.
*
* The permissions may be set if a file is created. Typically, an octal
* value is used, for example: 0775. The default value when calling
* newOutputStream is 0664.
*
* @param base the base object to write to
* @param mode flags controlling the writing
* @param permissions permissions of a file if one is created.
* @param bufferSize the size of buffer to use for buffered streams
* @param charset the character set to use when writing text streams
* @param extra the replacement character for unknown characters
* @returns a new output stream
* @throws NS_ERROR_INVALID_ARG when aBase is null
*/
nsISupports newOutputStreamWithArgs(in nsIVariant aBase,
in AString aMode,
in unsigned long aPermissions,
in unsigned long aBufferSize,
in AString aCharSet,
in AString aExtra);
The types and flags are based on the streams available in Mozilla already. In this case, I just use a type argument to the newInput/OutputStream methods to reduce the number of methods. Another possibility if support for optional arguments is possible is to have separate methods like newTextInputStream, newBinaryOutputStream, etc. Otherwise, the number of methods becomes unwieldly. A third possibility is to just add some properties to the returned stream objects to adjust the values. For instance, a charset property.
Files returned by the IO's newFile and newFileWithPath methods implement the nsIFile interface. The approach involves implementing nsIClassInfo on each of the file implementations. There are four, one for each of the Windows, Mac, Unix and OS2 platforms. This allows the interfaces to be flattened so that QueryInterface does not have to be called. It is important that a more scriptable version of a file implement nsIFile to be compatible with existing APIs that use files.
The nsIFile interface is similar in terms of its design to the Java File class. The interface of a file object is described below, but additional methods could be added as well to perform other useful functions.
File
{
const unsigned long NORMAL_FILE_TYPE = 0;
const unsigned long DIRECTORY_TYPE = 1;
void append(in AString node);
void appendRelativePath(in AString relativeFilePath);
void create(in unsigned long type, in unsigned long permissions);
void createUnique(in unsigned long type, in unsigned long permissions);
void copyTo(in nsIFile newParentDir, in AString newName);
void copyToFollowingLinks(in nsIFile newParentDir, in AString newName);
void moveTo(in nsIFile newParentDir, in AString newName);
void remove(in boolean recursive);
void reveal(); // open folder window in Finder/Explorer
void launch(); // launch/execute the file
attribute unsigned long permissions;
attribute unsigned long permissionsOfLink;
attribute PRInt64 lastModifiedTime;
attribute PRInt64 lastModifiedTimeOfLink;
attribute PRInt64 fileSize;
readonly attribute PRInt64 fileSizeOfLink;
attribute AString leafName;
readonly attribute AString path;
readonly attribute nsIFile parent;
readonly attribute nsISimpleEnumerator directoryEntries;
attribute PRBool followLinks;
readonly attribute PRInt64 diskSpaceAvailable;
boolean exists();
boolean isWritable();
boolean isReadable();
boolean isExecutable();
boolean isHidden();
boolean isDirectory();
boolean isFile();
boolean isSymlink();
boolean isSpecial();
nsIFile clone();
boolean equals(in nsIFile inFile);
boolean contains(in nsIFile inFile, in boolean recur);
attribute ACString persistentDescriptor;
ACString getRelativeDescriptor(in nsILocalFile fromFile);
void setRelativeDescriptor(in nsILocalFile fromFile,
in ACString relativeDesc);
void normalize(); // removes . and .. etc components
readonly attribute AString target; // not implemented on Mac
void initWithPath(in AString filePath);
void initWithFile(in nsILocalFile aFile);
// Windows specific features
AString getVersionInfoField(in string aField);
readonly attribute AString canonicalPath;
// Mac specific functions
readonly attribute PRInt64 fileSizeWithResFork;
void launchWithDoc(in nsILocalFile aDocToLoad, in boolean aBackground);
void openDocWithApp(in nsILocalFile aAppToOpenWith, in boolean aBackground);
boolean isPackage();
readonly attribute AString bundleDisplayName;
readonly attribute AUTF8String bundleIdentifier;
// these do nothing on MacOSX
void setFileTypeAndCreatorFromMIMEType(in string aMIMEType);
void setFileTypeAndCreatorFromExtension(in string aExtension);
// OS2 specific functions
nsIArray getFileTypes();
void setFileTypes(in ACString fileTypes);
PRBool isFileType(in ACString fileType);
void setFileSource(in AUTF8String aURI);
};
For the streams returned by the IO methods, it would be useful to reuse the existing stream interfaces if possible. However there are some issues with this:
- The nsIInputStream and nsIOutputStream methods have non-scriptable read methods which cannot be called or implemented in JavaScript. For input, a scriptable wrapper stream is used.
- The text input streams (nsIConverterInputStream, nsIUnicharLineInputStream, etc.) use out parameters which are more suitable for C++ but awkward in JavaScript. Also, bugs make implementing these interfaces unsuitable.
For reading text I would suggest some new methods, for reading strings and lines:
AString readString(in unsigned long aCount); boolean readLine();
These can be implemented by reading from the underlying stream and calling the methods of nsIScriptableUnicodeConverter to convert to and from characters into Unicode.
Something like the following:
InputStream
{
unsigned long available();
string read(in unsigned long aCount); // reads bytes without character/newline processing
AString readString(in unsigned long aCount); // reads text in the current charset
boolean readLine();
void close();
// don't need the NS_ prefixes but nsISeekableStream currently uses them
const long NS_SEEK_SET = 0;
const long NS_SEEK_CUR = 1;
const long NS_SEEK_END = 2;
// C-like seek/tell calls
void seek(in long whence, in long long offset);
long long tell();
void setEOF(); // truncate file at current point
// binary streams
PRBool readBoolean();
PRUint8 read8();
PRUint16 read16();
PRUint32 read32();
PRUint64 read64();
float readFloat();
double readDouble();
void readByteArray(in PRUint32 aLength,
[array, size_is(aLength), retval] out PRUint8 aBytes);
// multiplex streams
readonly attribute unsigned long count;
void appendStream(in nsIInputStream stream);
void insertStream(in nsIInputStream stream, in unsigned long index);
void removeStream(in unsigned long index);
nsIInputStream getStream(in unsigned long index);
}
OutputStreams would be of a similar design.
