Time Standards

[christopherchapman]

We need to figure out the best way to keep track of time for events. This is a discussion of time standards and how we will model them in the database and domain model and the rest service. We have many of the same problems as FamilySearch, because we are dealing with historic dates.

FamilySearch bases dates on the astronomical date, which is also known as a Julian date. It is a continuous count of days and fractions since noon Universal Time on January 1, 4713 BCE. Here is an example:

The Julian date for CE 2010 April 5 11:56:10.0 UT is
JD 2455291.99734
For more information you can see: http://aa.usno.navy.mil/data/docs/JulianDate.php

Other date formats can be calulated from this value. I don't know whether the FamilySearch date authority supports the float portion (the time of the day). Are we interested in keeping track of the time of an event as well as the date?

Edit: I confirmed that FamilySearch only supports dates. For family history, I imagine that researchers are not really concerned with anything more granular. Some events that we may want on the timeline happen very quickly sequentially, which would necessitate using time.

[russellhltn]

Are we interested in keeping track of the time of an event as well as the date?

In the case of multiple related events happening the same day, I think you'd want to have something in order to have the events presented in the proper sequence. Perhaps an event along the lines of "battle field ordination" alluded to during this past conference.

However, I hesitate to use time since the exact time of day may not be known. Entering a time will tend to cause it to become fact. Perhaps a structure that allows events to be described in relation to each other without an absolute time frame. There are certainly many parts of the Scriptures where time is not an absolute, but described in relation to another event. (Kind of like project planning software.) Changing the estimated Julian date for an event will cause all related events to move. It could actually simplify the construction of the time line and maintain accuracy.

[RossEvans]

We need to figure out the best way to keep track of time for events. This is a discussion of time standards and how we will model them in the database and domain model and the rest service. We have many of the same problems as FamilySearch, because we are dealing with historic dates.

I don't understand the context of "we" and "the database." Are you talking about one or more of the RESTful APIs that the Church is starting to provide to "community" developers?

[mkmurray]

I don't understand the context of "we" and "the database." Are you talking about one or more of the RESTful APIs that the Church is starting to provide to "community" developers?

This post is in the Church Historical Timeline sub-forum; it is one of the Community Projects on the wiki. That should be the context you need.

It is easy for us frequent members who always get around by doing "New Posts" to miss the context by not looking at the forum breadcrumb.

[christopherchapman]

However, I hesitate to use time since the exact time of day may not be known. Entering a time will tend to cause it to become fact. Perhaps a structure that allows events to be described in relation to each other without an absolute time frame. There are certainly many parts of the Scriptures where time is not an absolute, but described in relation to another event. (Kind of like project planning software.) Changing the estimated Julian date for an event will cause all related events to move. It could actually simplify the construction of the time line and maintain accuracy.

Using an abstract duration for related events could have some interesting possibilities...

The SIMILE timeline widget has the notion of an event that happens at a particular instant, as well as an event that has an imprecise start and/or end.

[christopherchapman]

I double checked with FamilySearch about the Date authority. It does not deal with time, only the date portion, and they have no plans to support time in the future. That makes sense, since the genealogy community is only concerned with dates. Most historic dates we will not have/need a time for, but I can possibly see a use for time for current events (ward timelines?).



There is, however, a good reason to still use the Date authority. All of the FamilySearch Authorities are essentially ontologies. For example the Place authority allows you to pass in something referring to a place and it will give you all the possible places, in a standard or authoritative form, that match. The date authority will take most any type of date format as an input and give you a standardized date for the response.



When we use the event API, and we want to see all events accross all timelines for a specific time, I am thinking of something like this:


http://timelines.lds.org/events?date={date}


The FamilySearch date authority could be queried to return a normalized date for many different types of input. This would make the API much more flexible. When we are dealing with an XML or JSON payload, we are already dealing with a programming language that would more likely have a standard date form already. A query parameter needs to be more flexible.

[rhusted-p40]

Should we consider using the Church History Department's date authority for start and end dates in the timeline and perhaps add a time column later? That way someone could enter the time of an event (it would need to be an optional field) if they wanted the timeline to sequence the events by date and time.

That's getting a bit ahead of ourselves though - to display time-sensitive events we have to alter the band for a given span of time... like the JFK example that MIT put together. For now, should we just use the Date Authority and put "time" as a parking lot item for the project? I'm thinking that will be easier and keep us from getting bogged down.

[christopherchapman]

Where the FamilySearch Date Authority really shines is in taking user input (in many different forms) and returning a canonized date. The more I think about it, the more I think that we should store dates as plain old dates (in some format) in the database, not really connected to the FamilySearch API. The Date Authority could be used for user input in the UI (Using ajax to return a canonized date for whatever they enter) or in the API as I mentioned in the previous posting. The database schema is going to need to be designed for quick searching by time. I don't know whether a Date or Float (storing the Julian Date) would be faster. We would probably have to profile it once we have a lot of dates in the database (like 10-20 million )

[christopherchapman]

Here are my thoughts as to how to store the event times in the database. We are designing for speed of filtering according to, first, the time of the event, and second, to the location of the event:

1. Store dates as char: very flexible, but we would need to load the whole table into memory, parsing all the dates into Date objects before we could filter by date.
2. Store dates as dates: In an Oracle db we can store dates from January 1, 4712 BC to December 31, 9999 AD. Indexing could be much faster than strings, because filtering could be done in SQL.
3. Store dates as numbers, using the astronomical date. See the first post in this thread for an explanation. Like with number 2, filtering could be done in SQL.

Does anyone have much profiling experience in Oracle? Would it be better to index a table on a date or a number?