1 Aug 2009 04:05
underflow is overflow
Richard Hennessy <rich.hennessy <at> verizon.net>
2009-08-01 02:05:20 GMT
2009-08-01 02:05:20 GMT
M:makelist(block([fpprec:500,s,t],s:bfloat(5.5b0*i*10^-11), t:f(5.5b0 * i * 10^-11, 2.749016986206715556222694148621332735422449427897640242607126584059308591366393146096574751526649826b-18, 2, 10), fpprec:10, bfloat([s,t])),i, -60*3, 60*3)$ (%i89) draw2d(points_joined=true,color=blue,point_type=dot,points(M)); float: floating point overflow converting 0.0b0 -- an error. To debug this try debugmode(true); I had fpprec set to 500 while computing y = f(x) and list M [[x1,y1],[x2,y2],[x3,y3],...] is using fpprec=10. Is this a bug? This is actually a case of underflow and I would be happy with 0b0 instead of the error. Rich
RSS Feed