This page contains a tutorial I wrote many years ago on Newton data storage. I had planned to do more but with the
cancellation of the Newton product etc. I never got round to it. It is here for anyone who is still working on
Newton products.
Note: If you find anything on this page that is wrong or if you know
of a better way to do something outlined here then let me know.
This page is my understanding of this subject and I may have bits
wrong! You can contact via email from here.
For more detailed discussion on Newton data storage consult the
following manuals:
Newton Programmers Guide: Chapter 11 (Apple Computer, Inc.)
Newton Programmers Reference: Chapter 9 (Apple Computer, Inc.)
Both of these, along with the other official manuals, are available for download in electronic format
by clicking on the links below. These manuals are in PDF format and are compressed with Stuffit.
Newton Programmers Guide
Newton Programmers Guide 2.1 Addendum
Newton Programmers Reference
NewtonScript Reference
Newton Interface Guidelines
Unlike traditional operating systems such as the MacOS and Windows NT there is no file system and therefore no files in the Newton Operating System. Data is instead stored in opaque collections like a database in entities known as soups. Each entry in a soup can be likened to a record in a database and each data member in an entry (known as a slot) can be likened to a field in a database. Unlike database records entries in a soup do not all have to have the same slots. To retrieve data entries from the soups you don't access them directly but rather send queries to the soups which return cursor objects. You then use the cursor object to get copies of individual entries in the soup. You can then do whatever you want with the copies. The original entries in the soups are not modified unless you overwrite them with your changed copy.
The actual soups themselves are held on the Newton in stores. Every Newton device has at least one store (the internal store) but they can also have other external stores on PCMCIA cards. On a Newton MessagePad 130 there can be one external store as there is one PCMCIA slot. Other devices such as the MessagePad 2000 which has two PCMCIA slots can have more external stores. So that you don't have to worry about which store your soup is on you can use a type of soup known as a union soup in your application. This is a system maintained virtual soup that is made up of all entries from a particular named soup no matter where the entries are physically stored. For Example - if you have two soups named "Match Results" (one on the internal store and one on an external store) to hold results of football matches you can treat them as one big soup containing all the entries by using a union soup. Union soups take into account any choices you make for default storage location and add new entries to a union soup in the member soup which resides on your default store (see AddToDefaultStoreXmit below).
To create a new union soup use the RegUnionSoup function to register a soup definition frame with the system. The system then uses this soup definition to create soups on stores as needed when new entries are written. So you don't have to worry about creating soups on each store. If you insert a PCMCIA card the system will create a member soup on it, if it needs to, using your soup definition and add it to the union soup. RegUnionSoup returns a union soup object to which you can send messages. If the union soup for the soup definition you pass to RegUnionSoup already exists it is returned and a new one is not created.
The following code fragment assigns a soup definition frame to the variable mySoupDef:
mySoupDef := {
// name = the system soup name
// it is composed of the name you give the soup with your dev signature appended
name: "mySoup:stormont",
//userName = the user visible soup name
userName: "My Application Soup",
//ownerApp = the owner application symbol
ownerApp: '|myApp:stormont|,
//ownerAppName = the user visible name off App that owns this soup
ownerAppName: "My Application",
//userDescr = user visible string describing soup
userDescr: "This soup is used by My Application",
//indexes = array of index specs - see below
indexes: [{
structure: 'slot, // required - index on a slot in a frame
path: 'aslot, // aslot = the actual slot you want to index on
type: 'string // the data type of aslot
order: 'ascending, // optional - can also be descending
sortID: nil // optional
}], // end of indexes spec
};// end of mySoupDef
You can include more than one index spec in the indexes slot of the soup definition frame. This means that you can add an index for each slot that you think you will need to sort the data on in your union soup. Accessing data in soups is quicker using indexes as you can search your soup using them without loading entries into the NewtonScript heap memory segment.
Once you have got your soup definition frame set up you pass it to RegUnionSoup. This either creates a new union soup or returns an existing one if it matches your soup definition. You call RegUnionSoup as follows:
// register a soup definition frame with the system and assign it to a
// variable
myUsoup := RegUnionSoup('|myApp:stormont|, mySoupDef);
The best place to do this is in the ViewSetUpFormScript method of the application base view. Doing it here makes the soup available for the entire time your application is running. This does however use up memory so you can opt to do it from some other place in the application. You can unregister the soup from the ViewQuitScript method of your application base view by calling the UnRegUnionSoup function as follows:
// Unregister a soup definition with the system UnRegUnionSoup (myUsoup.Name, '|myApp:stormont|); myUsoup := nil; // to allow garbage collection
To completely delete a soup you use the RemoveFromStoreXmit() function. This function does not work with union soups so you have to remove each member soup from each store individually. This is shown below:
// The next block deletes the soup from the internal store.
local tempSoup := GetStores()[0]:GetSoup("dataDump:stormont");
tempSoup:RemoveFromStoreXmit(nil);
Always ask before removing a soup. The user may still want to keep the data in it - for example if they are installing a later version of your application they will want to keep the soups created by the earlier version.
The only data objects that can be stored in soups are frames. Frames can hold any of the other NewtonScript data types so this is not a problem. To add a frame to a union soup you use the AddToDefaultStoreXmit function as follows:
// Define a frame with two slots
local myFrame := {name: "Ian Robinson", jobTitle: "Network Administrator"};
// make a copy of this frame by cloning it
local frameToAdd := clone(myFrame);
// add this frame to the union soup
// myUsoup is assumed to be a valid object returned from RegUnionSoup
myUsoup:AddToDefaultStoreXmit(frameToAdd, '|myApp:stormont|);
You have to use the clone function to make a copy of the frame you want to add to the soup as AddToDefaultStoreXmit destroys the frame passed to it during the process of converting it to a soup entry. So if you want to preserve the original frame pass in a copy. The second parameter for AddToDefaultStoreXmit is used in the change notification system which is described later.
To retrieve entries you send a Query to the union soup. The query returns a cursor object which contains a subset of the entries in the union soup. The members in this subset of entries is determined by the query specification frame or query spec which is passed to the query. You can query the soup for entries that have a particular slot, or for entries that have a slot which contains a certain value or range of values. In order to query a soup for a particular slot there must be an index for that slot. Remember that indexes are usually added in the soup definition frame. The simplest query uses nil as the value of the query specification frame. This returns all of the entries in the union soup. The order in which the entries is returned is not guaranteed to be in the same order that they were added to the soup. The following code fragment shows the simplest query:
// return cursor containing all entries in a union soup // myUsoup is assumed to be a valid union soup as returned by RegUnionSoup local allEntriesCursor := myUsoup:Query(nil);
To query a union soup to return all entries that contain a particular slot you pass the name of the slot in the query specs indexPath value. Remember that the slot you are querying on must be indexed. The entries in the cursor returned when querying on a slot name in this way are sorted on the value of the data in the named slot. The following code fragment shows a query to a soup that will return all entries containing a jobTitle slot:
// return cursor containing entries with jobTitle slot
nameCursor := myUsoup:Query({indexPath: 'jobTitle});
Most times you will not want to search the whole soup as this takes time. For example if you want to find values in a slot that are in a certain range you can specify a starting point and end point in the soup. To query on the jobTitle slot for jobs beginning with "n" you can supply a beginExclKey and an endExclKey value. The beginExclkey and endExclKey values are not included in the query. So if you want to find slots with a value beginning with "n" you supply a beginExclkey with a value of "m", the one before "n", and an endExclkey value of "o", the one after "n". This is shown in the following code fragment:
//return cursor containing entries beginning with n in the jobTitle slot
namesCursor := myUsoup:Query ({indexPath: jobTitle, beginExclKey: "m", endExclKey: "o"});
To select entries in a range that includes the starting and ending values use the beginKey and endKey values. These don't exclude the specified values but rather all those before and after.
//return cursor containing entries beginning with m, n and o in the jobTitle slot
namesCursor := myUsoup:Query ({indexPath: jobTitle, beginKey: "m", endKey: "o"});
You can mix the two types of specifier in a query spec. You can for example have a starting beginKey and an ending endExclKey or a starting beginExclKey and an ending endKey. You cannot however have a beginKey and beginExclKey in the same query. Similarly you cant have an endKey and endExclKey in the same query. For example another way to do the query shown previously which returned a cursor containing a jobTitle slot with values beginning with "n" would be as follows:
// return cursor containing entries beginning with n in the jobTitle slot
// Note that the beginKey is n - the value we want to start with and
// that the endExclKey is o - the value after n in alphabetical order. The
// endExclKey value is not included in the search so it starts and ends with n.
namesCursor := myUsoup:Query ({indexPath: jobTitle, beginKey: "n", endExclKey: "o"});
You can also pass a function in the query spec. The function can be passed in either of two methods. These are the validTest and indexValidTest methods. For the entries to be included in the cursor the function passed in these methods must return a non zero value. The validTest method is slower than the indexValidTest method as the former has to read each entry into memory before executing the function whilst the indexValidtest method does not. Both methods are slower than using the beginKey/beginExclkey and endKey/endExclKey methods outlined above. An example of the indexValidTest method is shown below:
// return cursor containing entries beginning with n in the jobTitle slot
// Note that BeginsWith() is a string function built into the NewtonScript language
// It takes two arguments - the first is a string to test and the second is the
// letter you want to see if the string begins with.
namesCursor := myUsoup:Query ({indexPath: jobTitle,
indexValidTest:
func(theString)
begin
return BeginsWith(theString, "n");
end
});
Another way to query a soup for entries is to do it for a particular string. You can query for word beginnings, entire words or for text sequences. These searches are not limited to a particular slot in an entry but rather add the entry to the cursor if any slot contains the string. To search a soup for entries that contain words beginning with a particular string you pass a query spec containing an words array:
// return cursor containing entries that have words beginning with "rob and "net"
// Because the words array contains two strings this query will only return entries
// that have words that begin with "rob" and other words that begin with "net".
// It will not return entries that only contain words that begin with one or other
// of the strings.
wordsCursor := myUsoup:Query ({words: ["rob", "net"]});
If you want to search for entire words and not just word beginnings you can add the entireWords slot to your query spec:
// return cursor containing entries that have the words "rob and "net" in them
// Because the words array contains two strings this query will only return entries
// that have both of the words. It will not return entries that only contain one
// of the words.
wordsCursor := myUsoup:Query ({words: ["rob", "net"], entireWords: true});
To search for any sequence of text in a soup you use the text method in the query spec. This works in much the same way as the examples shown above:
// return cursor containing entries that have the text "net" in them
// This will return all entries that contain the string anywhere in a slot
wordsCursor := myUsoup:Query ({text: ["net"]});
The query you send to a union soup returns an object known as
a cursor. This cursor contains all of the entries that matched
the query spec you supplied. There are various ways in which you
can manipulate the entries in the cursor. You can test the cursor
to see if it is valid by using the Status() method. This returns a 'valid symbol if there are no problems with the cursor. If the return
value is nil then there is a problem with the cursor. This problem could be
as simple as the current entry the cursor is pointing at is outside
the bounds of the cursor or it could be something more serious.
If a PCMCIA card is inserted into the Newton the soups on the
card are automatically included in your applications union soup.
If the soups on the card do not have the same indexPath as that
in your query spec or if the tagSpec in your query is not matched
by the tags in the soup on the card then your cursor becomes invalid
and the Status method will return nil. Another symbol that Status can return is 'missingIndex which is returned if the soup referenced by the cursor is missing
an index.
The current entry that the cursor points to can be returned by
using the Entry() method. You can move the current entry that the cursor points
to by using the Next method to advance it by one entry or you can step back one entry
by using the Prev method.
As shown previously to add an entry to a soup you can use the AddToDefaultStoreXmit function. To remove entries you can use the RemoveEntryFromSoup function. Most of the time you will be reading an entry from a soup, modifying its slots, and then writing it back to the soup. To write an entry back to the soup it came from and replace the original use the EntryChangeXmit function:
| Home | | | Newton | | | | | Canicula? |