Recently aarona brought this topic up again as a one of many subjects in this thread.
This thread is an atempt from my side to collect what have been said to form a good ground for futher discussion. Also the thread aarona created wasn't aimed for discussing the topics in that he/she brought up.
Old thread from 2004-07-14
oliver wrote:Here are my fileformat suggestions and reasoning. This list is not yet complete.
[ ... ]
Other file's should be normal unix style text files (Linefeeds only not carriage returns)
Map files could be XML if really required.
All these points are open for discussion naturally but I think we got a descent list with open standards here that should pose a problem.
jfs wrote:It shouldn't be too hard to make support for both DOS, UNIX and Mac style line endings, and if all three kinds were supported it would be easier for the developers and hackers, I think. Eventually the files could be converted at load time, so all text internally is stored with UNIX line endings.oliver wrote:Other file's should be normal unix style text files (Linefeeds only not carriage returns)
ChrisCF wrote:TBH, the style of line-ending shouldn't really come into it. what's the problem with using UNIX style endings (\n only) on Mac OS, or DOS style (\r\n) on *nix? Sensible text editors support all three, and a myriad of others. Oh, and the encoding will be UTF-8, without question.
As has been discussed before, we should ideally try and write as little code for these things as possible, making use of libraries instead. Somewhere in much earlier discussion, it was raised that with some libraries we can have an XML parser out-of-the-box, exactly as we can have the insides of a Vorbis player - the joys of well-defined standards
oliver wrote:Personal I prefer ASCII encoding cos it's the most basic you can get, But i think UTF-8 is quite compiatble with ASCII isn't it. I have to read into that.ChrisCF wrote:TBH, the style of line-ending shouldn't really come into it. what's the problem with using UNIX style endings (\n only) on Mac OS, or DOS style (\r\n) on *nix? Sensible text editors support all three, and a myriad of others. Oh, and the encoding will be UTF-8, without question.
Fileformats internally should be \n is what I ment and should be the preferred format. We should try to keep 'our' archive clean this way and not mix and match. same goes for C files etc. (On a side note, does gcc even take UTF-8 chars as input?) that the engine works with both formats I guess shouldn't be a problem, eventually anyway, but we gotta keep things clean and concistent in our code stash.
I agree that we should use as many _standard_ widley availble libraries as possible, but still keep the amount to a minium. Like not implementing 1000x features and use 10000x libs and go through library hell. Just because we write lesser code doesn't mean the game will then be less efficient. But I think I mentioned XML in my first post and I was reffering to using the XML(2) library for that yes.ChrisCF wrote:As has been discussed before, we should ideally try and write as little code for these things as possible, making use of libraries instead. Somewhere in much earlier discussion, it was raised that with some libraries we can have an XML parser out-of-the-box, exactly as we can have the insides of a Vorbis player - the joys of well-defined standards
I'm not going to quote more stuff about \n vs \r\n vs \r. If you are interested to read the rest of these posts check the orginal thread starting from here.ChrisCF wrote:It's compatible with ASCII, since ASCII is a 7-bit encoding. It's only when you introduce other things like the ISO-8859 range, which have no less than 15 (possibly more) different uses of the "extended ASCII" (8-bit range). It would only come into play in any files which contain strings (and we should use strings, not references).oliver wrote:Personal I prefer ASCII encoding cos it's the most basic you can get, But i think UTF-8 is quite compiatble with ASCII isn't it. I have to read into that.
Fair enough, in which case we need to decide on \n or \r\n now (though TBH I don't see the savings being that much using \n).but we gotta keep things clean and concistent in our code stash.
As for the issue of XML, my opinion is that we'd be better off with something like libxml2, rather than trying to implement everything from scratch, and finding too many bugs.
oliver wrote:If we go with xml, then Yeah i'd say deffinatly use libxml2 and for other things goes the same.
Old thread from 2005-04-14
jfs wrote:What's most interesting though, IMO, is object description formats. (Eg. how to describe a new vehicle to be available in the game.)
Many people immediately scream XML when it comes to this, but I think the most flexible solution would really be to use "scripted data files" for object descriptions! This is also one of the reasons I think a hybrid compiled/scripted language solution is good.
I think most people here have an idea of what can be done with GRF files in the TTDPatch alpha versions, and I'm sure everybody would want TE to be able to do the same (and more?), but in a much easier way Smile So an "object description script" would really consist of a number of functions which are called by the game engine whenever something specific happens. Eg. there is a function that fills a vehicle data structure when a new vehicle of this type is built, one that's called whenever a vehicle of this type is (attempted) added to a "vehicle chain" (eg. a train or a trailer to a lorry) and likewise one for removal, one when loading-status changes etc. etc. That would basically give ultimate freedom for vehicle designers. Also, for basic needs, a "template" could be supplied, so even non-programmers could easily create vehicles (though not as sophisticated ones) for the game.
Se the orginal post for the attachmentsZuu wrote:As a refference I have uploaded init_object.lua which defines all objects (which aren't that many ATM) in Junctioneer and LuaC.cpp which contains the C++ functions that is called from lua. The whole package can be obtained at http://www.sf.net/projects/junctioneer/ .jfs wrote:What's most interesting though, IMO, is object description formats. (Eg. how to describe a new vehicle to be available in the game.)
Many people immediately scream XML when it comes to this, but I think the most flexible solution would really be to use "scripted data files" for object descriptions! This is also one of the reasons I think a hybrid compiled/scripted language solution is good.
I think most people here have an idea of what can be done with GRF files in the TTDPatch alpha versions, and I'm sure everybody would want TE to be able to do the same (and more?), but in a much easier way So an "object description script" would really consist of a number of functions which are called by the game engine whenever something specific happens. Eg. there is a function that fills a vehicle data structure when a new vehicle of this type is built, one that's called whenever a vehicle of this type is (attempted) added to a "vehicle chain" (eg. a train or a trailer to a lorry) and likewise one for removal, one when loading-status changes etc. etc. That would basically give ultimate freedom for vehicle designers. Also, for basic needs, a "template" could be supplied, so even non-programmers could easily create vehicles (though not as sophisticated ones) for the game.
jfs wrote:On the scripting-question, I know Lua can be cumbersome to work in, but the API is still rather clean. I think there are some nicer C++ wrappers for it as well. (I wonder if a C++ wrapper can somehow be made with templates?)
That's why I also suggested looking at Python. I'm looking at its C/C++ API right now, and it seems to be nicer than Lua's at a first glance. I'll post some more after I've read most of the stuff there is in the docs
Edit:
Another thing, I think we should use a virtual filesystem approach to data files. So you can either have all files for eg. a vehicle inside a single packed file, or as separate files on the host filesystem. The engine shouldn't care about that. The only problem is what to do in case of a name clash (several files with the same path and name.)
Also, I've looked a bit more on the Python C API, and it does indeed seem rather nice. You don't need to do all that messing around with a stack like in Lua, but on the other hand, I'm not sure I have discovered how to properly call a function yet
Edit again:
Found it now. There's several ways of calling a function/method, but on of them is:Now, the really good thing here is the "format" and "..." arguments, you simply specify a kind of "format string" for the arguments the function takes, and then gives the arguments. It can automatically do conversion from basic C types and such. Functions can more or less return multiply values, in the sense that they can return a single touple that contains multiple values. Touples are also rather easy to deconstruct, I think.Code: Select all
PyObject* PyObject_CallFunction( PyObject *callable, char *format, ...)
Right now, I say we go for Python rather than Lua. Also, I think the "tables" and "meta-tables" concepts in Lua are confusing; Python seems to be more "normal" in most aspects.
New thread form Thursday this week ( 2006-06-08 )
aarona wrote:2. XML config files
--- Although the Design Documentation (ver 0.1.03 - Section 1.3) states we should use "Spreadsheets" or "Tab delimited files", I would strongly suggest useage of XML as our config file standard.
Here is some sample code which shows how easy it is to use XML provided you know all the things inside <> and ""'s, which would be provided in examples or documentation.
Suggestion: New thread (or continue the one from years ago...), poll, demonstration filesCode: Select all
<vehicle="Train">Choo-Choo <type>Steam</type> <speed>100</speed> </vehicle>
eis_os wrote:That really depends on how the engine works and how much different logic a designer will be able to do. XML can get quite complicate if you want to have things like variational cargo display, ages or other stuff selecting different sprites. And you should create a tag for the name of the object and not mix it.aarona wrote: 2. XML config files
Code: Select all
<vehicle="Train">Choo-Choo <type>Steam</type> <speed>100</speed> </vehicle>
Code: Select all
<name>Choo-Choo Patched</name> <description>A patched Choo Choo Engine</description>
aarona wrote:I agree, the point really is that it has to be done some way, so I believeeis_os wrote:XML can get quite complicate if you want to have things like variational cargo display, ages or other stuff selecting different sprites.
(from a developers point of view) XML would be *much* easier to implement over making your own. (Unless someone more motivated to spend time on a paser is willing to put their hand up ) It seems to me with regards to TTD-patch that people are either coders or graphics artists (exceptions, of course, do exist ) so I'm hoping that with TE we can have graphics artists who can do their own coding without having to learn too much.
XeryusTC wrote:2. XML works great if you want more programs to parse it, but most parsers are really slow. It would be better to just have your own format if only your application has to read it because it will probably be A LOT faster.
So, what to do from here?
So more discussion and elaboration if needed, and then a poll have to be set up.aarona wrote:Suggestion: New thread (or continue the one from years ago...), poll, demonstration files