headers
Jon Watte | 6 Oct 2010 00:03
Picon
Gravatar

Re: Data oriented programming

There are times when back-of-the-envelope calculations turn out to be wrong.

Or, perhaps more commonly, when implementation bugs cause systems to not behave the way you planned them.
CI builds catching performance problems as soon as they are checked into the code base seems quite preferrable to customers and reviewers catching bugs when you ship :-)
I don't see why you think adding performance tests to a CI tester is wrong? We have it, and have used it, and it's caught things for a long time. Even things that we planned well, but ended up having either unexpected side effects, or implementation problems.

Sincerely,

jw


--
Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable.



On Thu, Sep 30, 2010 at 1:58 PM, Fabian Giesen <ryg <at> gmx.net> wrote:
On 9/30/2010 1:36 PM, Richard wrote:
Your cache usage patterns (usually) depend on architectural decisions,
not implementation details. Any sort of integration testing (even if
you're doing CI) is too late a stage to detect architectural problems.
By that point you're already in trouble.

Not that I disagree, but if you're in trouble isn't it best to know as
soon as possible?  That's all I'm asserting.

Yes, it is best to know as soon as possible, and integration testing is most emphatically not "as soon as possible" for architectural decisions. Back-of-the-envelope calculations during the design stage are. Consider them unit tests for your design if you're hell-bent on viewing everything as an agile progress.

-Fabian 

_______________________________________________
Sweng-Gamedev mailing list
Sweng-Gamedev <at> lists.midnightryder.com
http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
Permalink | Reply | 24 comments |
headers
Fabian Giesen | 6 Oct 2010 00:33
Picon

Re: Data oriented programming

On 10/5/2010 3:03 PM, Jon Watte wrote:
> There are times when back-of-the-envelope calculations turn out to be wrong.
> Or, perhaps more commonly, when implementation bugs cause systems to not
> behave the way you planned them.
> CI builds catching performance problems as soon as they are checked into
> the code base seems quite preferrable to customers and reviewers
> catching bugs when you ship :-)
> I don't see why you think adding performance tests to a CI tester is
> wrong? We have it, and have used it, and it's caught things for a long
> time. Even things that we planned well, but ended up having either
> unexpected side effects, or implementation problems.

I didn't mean to suggest that you shouldn't test performance during 
testing, just that it shouldn't be your "first line of defense". 
Obviously you can't fix problems before you notice them, but a lot of 
these problems are entirely avoidable if you think things through early 
enough.

-Fabian
_______________________________________________
Sweng-Gamedev mailing list
Sweng-Gamedev <at> lists.midnightryder.com
http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
Permalink | Reply | 23 comments |
headers
Gregory Junker | 6 Oct 2010 05:48
Favicon

Re: Data oriented programming

> 
> I didn't mean to suggest that you shouldn't test performance during
> testing, just that it shouldn't be your "first line of defense".
> Obviously you can't fix problems before you notice them, but a lot of
> these problems are entirely avoidable if you think things through early
> enough.

Which can easily lead to "analysis paralysis"...

Greg 

_______________________________________________
Sweng-Gamedev mailing list
Sweng-Gamedev <at> lists.midnightryder.com
http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
Permalink | Reply | 22 comments |
headers
Tony Albrecht | 6 Oct 2010 06:12
Picon

Re: Data oriented programming

So you are recommending that you shouldn't think about the performance impact of your code before you write it?


I think you should. If you are working on code which is likely to have a performance impact on a system, then you should think carefully about the layout of data and code. Smart decisions at the start will save you a lot of time in the long run. I've worked on systems that are incredibly difficult to optimise due to uniformly slow code where a week spent fiddling around during the design of the system would have saved months down the track.

-Tony

On 6 October 2010 14:18, Gregory Junker <gjunker <at> dayark.com> wrote:
>
> I didn't mean to suggest that you shouldn't test performance during
> testing, just that it shouldn't be your "first line of defense".
> Obviously you can't fix problems before you notice them, but a lot of
> these problems are entirely avoidable if you think things through early
> enough.

Which can easily lead to "analysis paralysis"...

Greg

_______________________________________________
Sweng-Gamedev mailing list
Sweng-Gamedev <at> lists.midnightryder.com
http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
Permalink | Reply | 21 comments |
headers
Richard Fabian | 6 Oct 2010 11:52
Picon
Gravatar

Re: Data oriented programming

I'm with Tony on this one: Too many times "just write it and we'll optimise it later" has come to bite.


So many times it's been "just add a system to do this", and then three years later, we're not using it the way it was originally intended due to performance issues. For example, scripting languages that get too slow, get compiled to code, then get optimised further by doing trickery, then some random guy walks up and says, "so why not do this stuff in code anyway" and gets bashed for pointing out the painful but obvious.

Don't trap yourself in thinking that something that seems easy to use, can ever be made fast enough to USE. If you wait until your performance slot (say one year before release), then you're in for an almighty headache as you have to do awful things like change the way in which you even approach certain problems. Take some visibility algorithms for example, how many different ones are there? Portals, PVS, octree, BSP, blah blah blah..... don't think they're all there because someone thought "ooh, i'lll try doing it this way." No, they did it because in different circumstances, different algorithms do better than others, either due to layout of your levels, or layout of the code using it, or simply the access patterns into the structure. For anyone who's actually done a complete conversion of visibilty system from one to another, changing data structures and access techniques throughout a codebase: I feel your pain.

There are many different ways to lay out your data, some are performant, some are easy to manipulate... Not many are both.

Apply natural selection: If you don't care about performance from day 1, then manipulability wins over performance. Thus ends the tale.


On 6 October 2010 05:12, Tony Albrecht <tony.albrecht0 <at> gmail.com> wrote:
So you are recommending that you shouldn't think about the performance impact of your code before you write it?

I think you should. If you are working on code which is likely to have a performance impact on a system, then you should think carefully about the layout of data and code. Smart decisions at the start will save you a lot of time in the long run. I've worked on systems that are incredibly difficult to optimise due to uniformly slow code where a week spent fiddling around during the design of the system would have saved months down the track.

-Tony


On 6 October 2010 14:18, Gregory Junker <gjunker <at> dayark.com> wrote:
>
> I didn't mean to suggest that you shouldn't test performance during
> testing, just that it shouldn't be your "first line of defense".
> Obviously you can't fix problems before you notice them, but a lot of
> these problems are entirely avoidable if you think things through early
> enough.

Which can easily lead to "analysis paralysis"...

Greg


_______________________________________________
Sweng-Gamedev mailing list
Sweng-Gamedev <at> lists.midnightryder.com
http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com




--
fabs();
Just because the world is full of people that think just like you, doesn't mean the other ones can't be right.
_______________________________________________
Sweng-Gamedev mailing list
Sweng-Gamedev <at> lists.midnightryder.com
http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
Permalink | Reply | 16 comments |
headers
Mat Noguchi | 6 Oct 2010 18:53

Re: Data oriented programming

I think you overestimate anyone’s ability to predict the actual difficulties of shipping and underestimate the ability of programmers to bash a system into submission. Unless you are talking about the mean and not possibilities.

 

MSN

From: sweng-gamedev-bounces <at> lists.midnightryder.com [mailto:sweng-gamedev-bounces <at> lists.midnightryder.com] On Behalf Of Richard Fabian
Sent: Wednesday, October 06, 2010 2:53 AM
To: sweng-gamedev <at> midnightryder.com
Subject: Re: [Sweng-Gamedev] Data oriented programming

 

I'm with Tony on this one: Too many times "just write it and we'll optimise it later" has come to bite.

 

So many times it's been "just add a system to do this", and then three years later, we're not using it the way it was originally intended due to performance issues. For example, scripting languages that get too slow, get compiled to code, then get optimised further by doing trickery, then some random guy walks up and says, "so why not do this stuff in code anyway" and gets bashed for pointing out the painful but obvious.

 

Don't trap yourself in thinking that something that seems easy to use, can ever be made fast enough to USE. If you wait until your performance slot (say one year before release), then you're in for an almighty headache as you have to do awful things like change the way in which you even approach certain problems. Take some visibility algorithms for example, how many different ones are there? Portals, PVS, octree, BSP, blah blah blah..... don't think they're all there because someone thought "ooh, i'lll try doing it this way." No, they did it because in different circumstances, different algorithms do better than others, either due to layout of your levels, or layout of the code using it, or simply the access patterns into the structure. For anyone who's actually done a complete conversion of visibilty system from one to another, changing data structures and access techniques throughout a codebase: I feel your pain.

 

There are many different ways to lay out your data, some are performant, some are easy to manipulate... Not many are both.

 

Apply natural selection: If you don't care about performance from day 1, then manipulability wins over performance. Thus ends the tale.

 

 

On 6 October 2010 05:12, Tony Albrecht <tony.albrecht0 <at> gmail.com> wrote:

So you are recommending that you shouldn't think about the performance impact of your code before you write it?

 

I think you should. If you are working on code which is likely to have a performance impact on a system, then you should think carefully about the layout of data and code. Smart decisions at the start will save you a lot of time in the long run. I've worked on systems that are incredibly difficult to optimise due to uniformly slow code where a week spent fiddling around during the design of the system would have saved months down the track.

 

-Tony

 

On 6 October 2010 14:18, Gregory Junker <gjunker <at> dayark.com> wrote:

>
> I didn't mean to suggest that you shouldn't test performance during
> testing, just that it shouldn't be your "first line of defense".
> Obviously you can't fix problems before you notice them, but a lot of
> these problems are entirely avoidable if you think things through early
> enough.

Which can easily lead to "analysis paralysis"...

Greg

 


_______________________________________________
Sweng-Gamedev mailing list
Sweng-Gamedev <at> lists.midnightryder.com
http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com




--
fabs();
Just because the world is full of people that think just like you, doesn't mean the other ones can't be right.

_______________________________________________
Sweng-Gamedev mailing list
Sweng-Gamedev <at> lists.midnightryder.com
http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
Permalink | Reply | 7 comments |
headers
Fabian Giesen | 6 Oct 2010 20:53
Picon

Re: Data oriented programming

On 06.10.2010 09:53, Mat Noguchi wrote:
> I think you overestimate anyone’s ability to predict the actual
> difficulties of shipping and underestimate the ability of programmers to
> bash a system into submission. Unless you are talking about the mean and
> not possibilities.

Why do so many people on this list insist on turning this into a 
dichotomy? It's not. I completely agree that no amount of planning ahead 
will prevent you from having to change important parts of your codebase 
late in the project because your assumptions turned out to be wrong. I 
also agree trying to get everything perfect the first time round is a 
fool's errand. (And I doubt anyone with software development experience 
disagrees).

But I'm somewhat baffled that my suggestion of, in effect, "think before 
you type" is actually being met with resistance. Just because I can't 
fully predict the performance implications of a system in advance 
doesn't mean it's a waste of my time to think about it at all. And, in 
my experience anyway, *not* thinking about it in advance is a waste of 
mine and other people's time some months down the road.

Same thing with the testing. Again, it's not a dichotomy. Just because I 
think that testing is too late in the process to easily fix performance 
problems doesn't mean that performance testing is worthless, it just 
makes it a bad choice for your first line of defense.

-Fabian
_______________________________________________
Sweng-Gamedev mailing list
Sweng-Gamedev <at> lists.midnightryder.com
http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
Permalink | Reply | 4 comments |
headers
Mat Noguchi | 6 Oct 2010 21:02

Re: Data oriented programming

I was responding specifically to this:

> Apply natural selection: If you don't care about performance from day 1, 
> then manipulability wins over performance. Thus ends the tale.

Which was stated by Richard Fabian, not you.

MSN
-----Original Message-----
From: sweng-gamedev-bounces <at> lists.midnightryder.com
[mailto:sweng-gamedev-bounces <at> lists.midnightryder.com] On Behalf Of Fabian Giesen
Sent: Wednesday, October 06, 2010 11:53 AM
To: sweng-gamedev <at> lists.midnightryder.com
Subject: Re: [Sweng-Gamedev] Data oriented programming

On 06.10.2010 09:53, Mat Noguchi wrote:
> I think you overestimate anyone's ability to predict the actual
> difficulties of shipping and underestimate the ability of programmers to
> bash a system into submission. Unless you are talking about the mean and
> not possibilities.

Why do so many people on this list insist on turning this into a 
dichotomy? It's not. I completely agree that no amount of planning ahead 
will prevent you from having to change important parts of your codebase 
late in the project because your assumptions turned out to be wrong. I 
also agree trying to get everything perfect the first time round is a 
fool's errand. (And I doubt anyone with software development experience 
disagrees).

But I'm somewhat baffled that my suggestion of, in effect, "think before 
you type" is actually being met with resistance. Just because I can't 
fully predict the performance implications of a system in advance 
doesn't mean it's a waste of my time to think about it at all. And, in 
my experience anyway, *not* thinking about it in advance is a waste of 
mine and other people's time some months down the road.

Same thing with the testing. Again, it's not a dichotomy. Just because I 
think that testing is too late in the process to easily fix performance 
problems doesn't mean that performance testing is worthless, it just 
makes it a bad choice for your first line of defense.

-Fabian
_______________________________________________
Sweng-Gamedev mailing list
Sweng-Gamedev <at> lists.midnightryder.com
http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
_______________________________________________
Sweng-Gamedev mailing list
Sweng-Gamedev <at> lists.midnightryder.com
http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
Permalink | Reply | 0 comments |
headers
Richard Fabian | 6 Oct 2010 22:44
Picon
Gravatar

Re: Data oriented programming

I'm not sure I actually understand what you've written there (especially the bit about "mean and not possibilities",) but I think my point is orthogonal to shipping issues. Normally, a game has either good performance or bad, and that doesn't really affect whether or not it ships. Normally it only affects what it is shipped with.


Bashing a system into submission is something I have done a number of times, but sometimes we've had to go after smaller fish to bash as the real large fish is just too hard to make good. If you don't consider the difficulty of future optimisations, then you run the risk of having to redo part of the project.

To me, it sounds like you're happy ripping systems out to replace them, and that would make sense if you're working on the same kind of code base game after game, things change, but everything kinda stays the same. That's great, you can throw away whole ways of working and introduce new stuff and see how it works, but when you're doing multiple projects in the same studio without much overlapping tech or at least without much overlapping requirements, your estimated schedules are wild and inaccurate because each project is like you've never done a project like it before. Putting performance estimations off and relying on late in the game optimisation will get you a game, but not one that is great. Something will suffer.

One last point, performance is one of those things that fuels it's own fire. Like asset build optimisations or iteration time reduction. Every time you make one of these things better, it makes the whole studio run faster.


On 6 October 2010 17:53, Mat Noguchi <matthewn <at> bungie.com> wrote:

I think you overestimate anyone’s ability to predict the actual difficulties of shipping and underestimate the ability of programmers to bash a system into submission. Unless you are talking about the mean and not possibilities.

 

MSN



--
fabs();
Just because the world is full of people that think just like you, doesn't mean the other ones can't be right.
_______________________________________________
Sweng-Gamedev mailing list
Sweng-Gamedev <at> lists.midnightryder.com
http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
Permalink | Reply | 1 comment |
headers
Gregory Junker | 6 Oct 2010 22:48
Favicon

Re: Data oriented programming


So you are recommending that you shouldn't think about the performance
impact of your code before you write it?

I don't see how you could possibly have read that into my comments.

Greg

_______________________________________________
Sweng-Gamedev mailing list
Sweng-Gamedev <at> lists.midnightryder.com
http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
Permalink | Reply | 3 comments |

Gmane