OTTD Unlimited?

[BobDendry]

Everyone who has ever read NFO knows that it is basically a write-only language.

And as DaleStan truthfully points out, you can't learn nfo by looking at others nfo-code, in contrast to lua which has a somewhat clear syntax.

You seem to be jumping to a conclusion here. It's quite easy to pick up NFO(note the correct capitalisation) by looking at others' code. While I mostly code by GRFMaker, I learnt what I do know about NFO by looking at others code. Sorry, your logic is flawed.

[Expresso]

NFO is bytecode (which makes it unnescesary complicated to learn and prone to bugs). Yes, I'm aware there are tools which make creating NFO files easier, but I believe those are merely a workaround for a broken standard (I consider binary config files broken by default).

LUA is a complete scripting language, which would seem like overkill to me. However, XML seems seems just fine for the job and provides a lower learning curve as LUA and a lot lower learning curve as NFO.

While XML makes for larger files, it'll be a lot more readable as NFO files. As a result creating grm's will be more newbie friendly as createing grf's (the newbie won't be scared away by wierd bytecode). A newbie might actually start building XML-grm's and help expand openttd.

Yes, it'll be a lot of work to implement. But it'll be worth it. No, I won't work on that pile of spaghetti code which is openttd. But I am willing to help out with the conversion of the specification docs.

[Haukinger]

How fast is nfo btw ? It's run on a virtual machine, too, isn't it ?

I think we've found the winner to the "The Most Stupid Post of the Month" competition. You should keep your mouth shut on things you know nothing about.

Code: Select all

void LoadNewGRF(uint load_index, uint file_index)
{
	GrfLoadingStage stage;

	InitializeGRFSpecial();

	ResetNewGRFData();
[...]

What does this function do ? It runs a vm with a certain state and a certain instruction set.

Those instructions can have impact outside the vm, of course, because it would be a pretty much useless scripting engine if it couldn't interact with the game.

[Haukinger]

LUA is a complete scripting language, which would seem like overkill to me. However, XML seems seems just fine for the job and provides a lower learning curve as LUA and a lot lower learning curve as NFO.

NFO is a complete language, too. Just putting NFO in an xml-file doesn't help.
Think of this (no valid code, for illustration purposes only):

Code: Select all

<Action 01>
<Parameter 01>17</Parameter 01>
<Parameter 02>5</Parameter 02>
<Parameter 03>11</Parameter 03>
<Parameter 04>2</Parameter 04>
</Action 01>

It's more or less the same. Xml serves as a language to desribe a document, a trainset for example, and it's fine for this. But you can't program behaviour in xml. That's where lua comes into play.

[Haukinger]

Everyone who has ever read NFO knows that it is basically a write-only language.

And as DaleStan truthfully points out, you can't learn nfo by looking at others nfo-code, in contrast to lua which has a somewhat clear syntax.

You seem to be jumping to a conclusion here. It's quite easy to pick up NFO(note the correct capitalisation) by looking at others' code. While I mostly code by GRFMaker, I learnt what I do know about NFO by looking at others code. Sorry, your logic is flawed.

Could you be more precise about "quite easy" ? Do you mean easier than reading the ottd's source code for example ? Or easier than experimenting with NFO without looking at others' code ? I'd agree to the latter
Perhaps I should have written "can't easily learn", but to me that's nitpicking. Perhaps I'm different from all other computer users, but I can read and understand (almost all) plain text code better than (almost all) bytecode.

[peter1138]

What does this function do ? It runs a vm with a certain state and a certain instruction set.

Funny, I thought they just cleared out and initialized some data. Maybe I'm wrong, I only wrote it...

There is no VM involved in handling this; just a few functions with switch blocks.

[Haukinger]

just a few functions with switch blocks.

That's what I call a virtual machine. The state may be minimal, i.e. an array of bytes used as input and a pointer to the active position in the array.
A vm simulates a cpu that can understand machine code the actual cpu can't understand. In ottd, the vm understands nfo-bytecode...

[BobDendry]

Could you be more precise about "quite easy" ? Do you mean easier than reading the ottd's source code for example ? Or easier than experimenting with NFO without looking at others' code ? I'd agree to the latter

No, my definition of "quite easy" conforms to the former option.

[Brianetta]

just a few functions with switch blocks.

That's what I call a virtual machine. The state may be minimal, i.e. an array of bytes used as input and a pointer to the active position in the array.
A vm simulates a cpu that can understand machine code the actual cpu can't understand. In ottd, the vm understands nfo-bytecode...

That's an important point, and too important to ignore. Its state is minimal. It's a tiny virtual machine, tightly integrated into the game. There's no issues with interfaces, because the machine was written into OpenTTD with OpenTTD in mind, and the same will undoubtedly go for the Patch. LUA is a big, general purpose language. KUDr is, I believe, contemplating its use for a competitor AI. Newgrf was devised by Patchman as a way to integrate arbitrary game information and code into the game without losing performance.

XML and LUA cannot hope to perform as well as NFO with inline sprite information can (correct me if that's not quite what a newgrf is). We don't need a relatively heavy general purpose VM in the game. We need an integrated, lightweight, optimised one. One that was written with a sprite- and tile-based transport game in mind. Funny thing is, we have that. What's there works. It isn't broken, and it doesn't need fixing.

The only other thing we need is a full-spec NFO compiler. It might compile LUA scripts to NFO, or it might do Scheme, or Python, or a custom language. That shouldn't matter, and it's a choice that is entirely made by the compiler's designer. Heck, we could have two competing compilers with different languages.

Then we could even make the source of GPL newgrfs available... amazing.

[Nappe1]

just a few functions with switch blocks.

That's what I call a virtual machine. The state may be minimal, i.e. an array of bytes used as input and a pointer to the active position in the array.
A vm simulates a cpu that can understand machine code the actual cpu can't understand. In ottd, the vm understands nfo-bytecode...

I would call that as Virtual State Machine which means completely different thing than Virtual Machine

but that's me. Terms get easily mixed up these days when there's so much new terms coming out.

[Haukinger]

That's an important point, and too important to ignore. Its state is minimal. It's a tiny virtual machine, tightly integrated into the game. There's no issues with interfaces, because the machine was written into OpenTTD with OpenTTD in mind, and the same will undoubtedly go for the Patch. LUA is a big, general purpose language. KUDr is, I believe, contemplating its use for a competitor AI. Newgrf was devised by Patchman as a way to integrate arbitrary game information and code into the game without losing performance.

If I'm wrong, please correct me, but I think the newgrf-thing is older than ottd. Obviously, the ttdpatch-developer don't have access to the source code, so they had to cope with what the executable offers and what can be hacked into it. The newgrf format is not as it is because it is clever or fast or has any other hidden benefits, but because the ttdpatch-executable understands it.

XML and LUA cannot hope to perform as well as NFO with inline sprite information can (correct me if that's not quite what a newgrf is). We don't need a relatively heavy general purpose VM in the game. We need an integrated, lightweight, optimised one. One that was written with a sprite- and tile-based transport game in mind. Funny thing is, we have that. What's there works. It isn't broken, and it doesn't need fixing.

You're saying that whatever newgrf for whatever entity in ttdpatch that works correctly in ttdpatch will work correctly in ottd ?

As for performance, most of the nfo-processing seems to be done upon loading the newgrf, and that's effectively free time.
Noone says that a lua script is not allowed to create a map<cargo_id,spriteset> (living in the game, not the vm) by recolouring a single spriteset depending on the types of cargoes that exist. But it does so upon loading, so it is not required to do in realtime. The game then uses map<cargo_id,spriteset>::operator[] to lookup the correct spriteset (std::map is quite fast, O(log N), I think - but you could use any data structure here, I just want to illustrate things).
This could be the default behaviour (in the xml-file a lua-script is specified that creates a map<>). However one strange trainset could define a car whose sprite depends not on the cargo but on the actual background bitmap. So it would say (in xml, essentially it's going to be kind of a callback-table) "hey, I don't create precoloured sprites, call this lua-script instead and pass it a reference to the bitmap your painting everytime you want to draw me". This object would draw A LOT slower than other objects, but it could draw anything. This will be needed quite rarely, because there can be different prerendering schemes as new behaviour in newgrfs shows up, only the newest, most innovative ones would be self-drawing.

The only other thing we need is a full-spec NFO compiler. It might compile LUA scripts to NFO, or it might do Scheme, or Python, or a custom language. That shouldn't matter, and it's a choice that is entirely made by the compiler's designer. Heck, we could have two competing compilers with different languages.

Then we could even make the source of GPL newgrfs available... amazing.

That would solve some (most ?) problems, but it means stopping half way. You can still only do what newgrfs can do in ttdpatch.

[Haukinger]

I would call that as Virtual State Machine which means completely different thing than Virtual Machine

but that's me. Terms get easily mixed up these days when there's so much new terms coming out.

AHHHHHHHHHHHHHHHHHHHHHHHHHHHH

Enlighten me, then, but preferably by PM, as what the fundamental difference is
I always thought that every vm has a state, as every real machine has, too, perhaps even the universe, if seen as "machine", quantum physics left aside.

Back to game-programming - you have the game, and you have, plugged into that, a vm that executes a scripting language. The vm and the game interact, so the vm is not isolated and cannot live on itself, at least it can't do anything productive.
The vm can be anything - a switch-case- and callback-table-driven nfo-interpreter, lua or a full-fledged java-vm, even another pc connected by some kind of network and programmed in fortran or apl
However, you have scripting at all in the first place to have two levels of programming requiring different levels of expertise. The game (C, C++, java or anything else) is normally quite complex and difficult to understand and extend, while the scripts (lua, java, even nfo ) are designed to be easy to understand and write, but therefor can do less than the game, and only things the game allows them to do. The scripts can be written by users of your game, without the need for the sourcecode or a compiler or even an ide.
In an open-source game as ottd the borders between developer and user are not as clean and sharp as in a commercial game, but they're still there. Or should be, because having the users to be developers limits the userbase...

[Darkvater]

How fast is nfo btw ? It's run on a virtual machine, too, isn't it ?

I think we've found the winner to the "The Most Stupid Post of the Month" competition. You should keep your mouth shut on things you know nothing about.

Very useful post WhiteHand... Can you limit such quality posts to the off-topic section? Thank you.

Face it; nobody (the current openttd developers at least) is going to write support for another scripting language into openttd to replace or live side by the grf specification. As Brianetta said the only viable option is making an X > NFO compiler that has the same power as NFO. If you look at the NFO wiki (http://wiki.ttdpatch.net/tiki-index.php ... phicsSpecs), you can see it can change (almost) every aspect of the game. If OTTD implements those specs then yes everything working in ttdpatch will work correctly in ottd.

I am sure if someone is to write an easier to use language than NFO with the same power and which can be converted to NFO many people will be greatful to that person and even happily use that format instead of hex-ing.

But again, let me stress this; none of us is prepared to go through the addition of another mod-language into ottd and deal again with all its bugs, features and quirks.

[Haukinger]

If OTTD implements those specs

That's exactly my starting point - and it boils down to that. Which option promises more gain/work - completing the implementation of those specs or leaving the newgrf support as it is and implementing a cleaner interface ?
However, with the given chances of patches coming into trunk, I'm not investing any time in this matter. Be it otherwise, there might soon be hhxl-grfs (Haukingers-Horrible-Xml-Lua )...

That avatar of yours is cool !

[Brianetta]

Newgrf was devised by Patchman as a way to integrate arbitrary game information and code into the game without losing performance.

If I'm wrong, please correct me, but I think the newgrf-thing is older than ottd. Obviously, the ttdpatch-developer don't have access to the source code, so they had to cope with what the executable offers and what can be hacked into it. The newgrf format is not as it is because it is clever or fast or has any other hidden benefits, but because the ttdpatch-executable understands it.

TTD Patch developers have access to Patch's source code, and don't "hack" anything into it (any more). Newgrf wasn't forced by some limitation of the executable. it was designed by Patchman, and has a written specification. The TTDPatch wiki has the full spec.

Funny thing is, we have that. What's there works. It isn't broken, and it doesn't need fixing.

You're saying that whatever newgrf for whatever entity in ttdpatch that works correctly in ttdpatch will work correctly in ottd ?

No. I'm saying that newgrf works. OK, OpenTTD hasn't implemented the entire spec, but it will in time. This is important for those of us who wish to use the extensive library of existing, available newgrfs, not just those who want to be able to make new ones. Don't think that OpenTTD and Patch developers don't communicate with each other about this stuff, either. When new additions to the specification happens, many people are involved.

The only other thing we need is a full-spec NFO compiler. It might compile LUA scripts to NFO, or it might do Scheme, or Python, or a custom language. That shouldn't matter, and it's a choice that is entirely made by the compiler's designer. Heck, we could have two competing compilers with different languages.

Then we could even make the source of GPL newgrfs available... amazing.

That would solve some (most ?) problems, but it means stopping half way. You can still only do what newgrfs can do in ttdpatch.

No, because newgrf is not static. NFO can be extended, and has been many times. It remains backward compatible, and as I said before, the two dev teams do communicate. The Patch team understand that NFO is effectively a shared specification, and OpenTTD input isn't ignored.

[Haukinger]

TTD Patch developers have access to Patch's source code, and don't "hack" anything into it (any more). Newgrf wasn't forced by some limitation of the executable. it was designed by Patchman, and has a written specification. The TTDPatch wiki has the full spec.

Ok, learned a new thing.

No. I'm saying that newgrf works. OK, OpenTTD hasn't implemented the entire spec, but it will in time. This is important for those of us who wish to use the extensive library of existing, available newgrfs, not just those who want to be able to make new ones. Don't think that OpenTTD and Patch developers don't communicate with each other about this stuff, either. When new additions to the specification happens, many people are involved.

That's in fact the only reason at all I see for ottd to support newgrf - the exisisting newgrfs. Besides, see my previous post.

No, because newgrf is not static. NFO can be extended, and has been many times. It remains backward compatible, and as I said before, the two dev teams do communicate. The Patch team understand that NFO is effectively a shared specification, and OpenTTD input isn't ignored.

That means a nfo-compiler would have to be constantly updated, as nfo changes. That's why I mentioned lua, it already extends as far as any (scripting) language can extend. I'd design scripting the other way round - first let newgrfs do whatever they like, and then add specialized callbacks for those features that are used most or those that consume the most vm-cycles.

[Brianetta]

No, because newgrf is not static. NFO can be extended, and has been many times. It remains backward compatible, and as I said before, the two dev teams do communicate. The Patch team understand that NFO is effectively a shared specification, and OpenTTD input isn't ignored.

That means a nfo-compiler would have to be constantly updated, as nfo changes. That's why I mentioned lua, it already extends as far as any (scripting) language can extend. I'd design scripting the other way round - first let newgrfs do whatever they like, and then add specialized callbacks for those features that are used most or those that consume the most vm-cycles.

Only for new features. But then, new functions would have to be added to LUA too. Remember, the NFO spec is backwards compatible, so if you don't need new features, the old compiler will still be fine for old code. LUA would be as useless for scripting carrier pigeons into the game as NFO, without either of them being extended to support this new concept.

[Quark]

… newgrf is not static. NFO can be extended, and has been many times. It remains backward compatible, and as I said before, the two dev teams do communicate. The Patch team understand that NFO is effectively a shared specification, and OpenTTD input isn't ignored.

So-so Let's make some extensions to enable automatic distributing of newgrf through OTTD servers.

[Haukinger]

No. Lua could be allowed to do everything from the beginning.
Just for illustration purposes, not meant to be a real specification:

Imagine a newgrf like this:

Code: Select all

<Vehicle name="Carrier Pigeon">
<CreateTypeScript name="createpigeontype.lua" />
<CreateInstanceScript name="createpigeoninstance.lua" />
</Vehicle>

The script createpigeontype.lua registers a new type of vehicle with ottd (calling ottd's vehicletypeid registervehicletype( vehicletypeid basetype )) to get a new id for the carrier pigeon. Then it calls ottd's void registercallback( vehicletypeid, callbackid, callback ) repeatedly to register it's functions with ottd. These include on_draw( bitmap ), on_arrival( station ), on_departure( station ), on_tick( clock ).
Perhaps it also wants to change some of the properties of its vehicletype, so that it only lands on pigeonries instead of airports (if it inherits from vehicletype aircraft). It calls get_vehicletype(myid):setlegalstations( pigeonryid ). Pigeonries have to be defined somewhere else in the newgrf.
Whenever ottd wants to create a carrier pigeon, for example if the user buys one, it calls createinstance() in createpigeoninstance.lua which returns a vehicle struct with properly setup sprites...
You get the point.

[Brianetta]

Well, I wasn't envisioning inheriting from aircraft. I was envisioning a message carrier between headquarter rooftops, but there we go. Thing is, you're proposing defining an entire vehicle class in LUA, not C or (for Patch) assembly. Which is going to be faster? Assembly. We don't want to have such a general purpose language as that, as I've stated before. It can't perform as well as... just not having one.