{"id":38,"date":"2011-10-23T00:57:47","date_gmt":"2011-10-23T00:57:47","guid":{"rendered":"http:\/\/minkhollow.ca\/books\/?page_id=38"},"modified":"2014-07-21T17:36:22","modified_gmt":"2014-07-21T17:36:22","slug":"9-2","status":"publish","type":"page","link":"http:\/\/minkhollow.ca\/books\/?page_id=38","title":{"rendered":"9: Verification & Validation"},"content":{"rendered":"

\u00a0Chapter 9<\/h1>\n

Here’s were we will put color versions of the images, updates, and other extras.<\/p>\n

How do you ensure that the simulation you design simulates your model faithfully? In order to have confidence that the key elements of the simulation are faithful implementations of the original system it is necessary to verify both the model and the resultant simulation. This chapter will explain the process and examine several example approaches.<\/p>\n

\"wordl-ch-09-mar-17\"<\/a><\/p>\n

    \n
  1. \n
    What is Verification & Validation in a Simulation or a Game?<\/div>\n<\/li>\n
  2. \n
    How Do We Know There\u2019s A Problem?<\/div>\n
      \n
    1. \n
      Collecting Confirming Data<\/div>\n<\/li>\n<\/ol>\n<\/li>\n
    2. \n
      Verification<\/div>\n
        \n
      1. \n
        Code Verification<\/div>\n<\/li>\n
      2. \n
        Solution Verification<\/div>\n<\/li>\n<\/ol>\n<\/li>\n
      3. \n
        Validation<\/div>\n
          \n
        1. \n
          Conceptual Model Validation<\/div>\n<\/li>\n
        2. \n
          Face Validity<\/div>\n<\/li>\n
        3. \n
          Internal Validity<\/div>\n<\/li>\n
        4. \n
          Event Validity<\/div>\n<\/li>\n
        5. \n
          Sensitivity Analysis<\/div>\n<\/li>\n
        6. \n
          Historical validation<\/div>\n<\/li>\n
        7. \n
          Predictive Validation<\/div>\n<\/li>\n
        8. \n
          Extreme Condition\/Degenerate Tests<\/div>\n<\/li>\n<\/ol>\n<\/li>\n
        9. \n
          Validation and Verification of Games<\/div>\n
            \n
          1. \n
            Play Testing<\/div>\n<\/li>\n
          2. \n
            Effectiveness<\/div>\n<\/li>\n<\/ol>\n<\/li>\n
          3. \n
            Summary<\/div>\n
              \n
            1. \n
              Concepts<\/div>\n<\/li>\n
            2. \n
              Terminology<\/div>\n<\/li>\n<\/ol>\n<\/li>\n
            3. \n
              References, Notes & Further Resources<\/div>\n<\/li>\n<\/ol>\n

              Images<\/h2>\n

              *Please note: all images are copyrighted, permission to use an image must be obtained from Wiley<\/a> (and possibly also the original sources).<\/p>\n

               <\/p>\n

              \"\"<\/a>
              Figure 9.1: – Position of the weight plotted as a function of time, as generated by the first version of the Simple Harmonic Motion simulation.<\/figcaption><\/figure>\n
              \"\"<\/a>
              Figure 9-2: How a debugger looks while running. The computer program is in source code form and is executed statement-by-statement. The yellow arrow indicates the current statement, while red dots marke statements where execution will pause so that the programmer can examine variables.<\/figcaption><\/figure>\n

              Single Server queuing model – SIMULA code, exponential arrivals\/departures <\/span><\/p>\n

              comment
              \nSingle server queueing model – exponential distribution.<\/p>\n

              This is one half of the UNIX arrival process model. This is
              \nthe ‘control’ – the usual assumption on the arrival process is that
              \ninterarrival times are exponentially distributed. This model
              \nruns for two simulated weeks, and collects queue stats and metrics
              \nto compare with the next model, using UNIX interarrival times;<\/p>\n

              simulation begin<\/p>\n

              ref (head) q;\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0comment The system queue;
              \nreal u1,u2;
              \nreal lastidle, idtime, arrtsum, rtime, qtime, sizeint, ls;
              \nreal mitime, mstime, q1samp, q0samp, sersum, q2samp,t1,t2;
              \ninteger nidle, narr, nsers, nsamps, V1, V2;
              \ninteger\u00a0 sizeofq, nrsamps, nzsamps, ns;
              \nreal a, b, c, d, e, f,x1,x2;<\/p>\n

              link class arrentry;
              \nbegin
              \nreal arrtime;
              \nend;<\/p>\n

              process class arrprocess;
              \nbegin
              \nref(arrentry) b;
              \nreal a;<\/p>\n

              while true do
              \nbegin
              \nb :- new arrentry;\u00a0\u00a0 \u00a0comment\u00a0 Create a new queue entry;
              \nb.arrtime := time;\u00a0\u00a0 \u00a0comment\u00a0 save the arrival time;
              \nb.into(q);\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0comment\u00a0 Put this entry into the queue;
              \nsizeint := sizeint + sizeofq*(time-ls); ns := ns+1;
              \nls := time;
              \nsizeofq := sizeofq + 1;
              \nif server.idle then
              \nbegin
              \nnidle := nidle + 1;
              \nidtime := idtime + (time-lastidle);
              \nactivate server after current;
              \nend;
              \na := arrtime;
              \narrtsum := arrtsum + a;
              \nnarr := narr + 1;<\/p>\n

              hold ( arrtime );
              \nend;
              \nend;<\/p>\n

              process class serve;
              \nbegin
              \nref(arrentry) x;
              \nreal a, b, c;<\/p>\n

              while true do
              \nbegin
              \nif q.empty then begin
              \nlastidle := time;
              \npassivate;
              \nend;<\/p>\n

              x :- q.first;
              \nq.first.out;
              \na := sertime;
              \nnsers := nsers + 1;
              \nsersum := sersum + a;
              \nhold(a);
              \nsizeint := sizeint + sizeofq*(time-ls);
              \nls := time; \u00a0\u00a0 \u00a0ns := ns + 1;
              \nsizeofq := sizeofq – 1;
              \nrtime := rtime + (time-x.arrtime);
              \nnrsamps := nrsamps + 1;
              \nqtime := qtime + (time-x.arrtime-a);
              \nend;
              \nend;<\/p>\n

              process class demon;
              \nbegin
              \ninteger i;<\/p>\n

              while true do
              \nbegin
              \nhold (60);\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0comment Sample every 1 Min;
              \nnsamps := nsamps + 1;
              \ni := sizeofq;
              \nq0samp := q0samp + i;
              \nif (i > 2) then
              \nbegin\u00a0\u00a0 \u00a0q1samp := q1samp + (i-1);
              \nnzsamps := nzsamps + 1;
              \nend;<\/p>\n

              if i>1 then i := i-1
              \nelse i := 0;
              \nq2samp := q2samp + i;
              \nend;
              \nend;<\/p>\n

              real procedure sertime;
              \nbegin
              \nsertime :=\u00a0 negexp (1.0\/mstime, V2);
              \nend;<\/p>\n

              real procedure arrtime;
              \nbegin
              \nreal x1,x2;<\/p>\n

              if (time > t2) then
              \nbegin\u00a0\u00a0 \u00a0t1 := t1 + 900;\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0comment Add 15 minutes;
              \nt2 := t2 + 900;
              \nx1 := inreal;\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0comment The time period;
              \nx1 := inreal;\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0comment The mean;
              \nx2 := inreal;\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0comment The standard deviation;
              \nx2 := x2 * x2;\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0comment Variance;
              \ncomment\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0outtext (”\u00a0 Mean is “) outreal (x1,4,12)
              \nouttext (” SD is “) outreal(x2,4,12)
              \noutimage;<\/p>\n

              getmeans(x1,x2);
              \ncomment\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0outtext (“U1 is “) outreal (1.0\/u1,4,12)
              \nouttext (“U2 is “) outreal (1.0\/u2,4,12)
              \noutimage;
              \nend;
              \nx1 := hypoexp (u1, u2, V1);\u00a0\u00a0 \u00a0comment Assume hypoexponential;
              \narrtime := x1;
              \nend;<\/p>\n

              real procedure hypoexp (u1,u2, V2);
              \nname V2;
              \nreal u1,u2;\u00a0\u00a0 \u00a0integer V2;
              \nbegin
              \nhypoexp := negexp (u1, V2) + negexp (u2, V2);
              \nend;<\/p>\n

              procedure getmeans (x1,x2);
              \nreal x1,x2;
              \nbegin
              \nreal a,b,c,d,e,f;<\/p>\n

              a := 2.0*x2 – x1*x1;
              \nif (a < 0.0) then
              \nbegin\u00a0\u00a0 \u00a0outtext (“** Complex solution for mean\/sd. “);
              \noutreal (time, 4, 12);\u00a0 outreal(x1,4,12); outreal(x2,4,12);
              \noutreal (a,4,12); outimage;
              \nu1 := 2.0\/(x1);\u00a0\u00a0 \u00a0u2 := u1;
              \nend else begin
              \na := sqrt (a);<\/p>\n

              b := (x1 + a)\/2.0;
              \nc := (x1 – a)\/2.0;
              \nif (b<0.0) and (c<0.0) then
              \nbegin\u00a0\u00a0 \u00a0outtext (“** Negative means “);
              \noutreal (time, 4, 12);\u00a0 outreal(x1,4,12); outreal(x2,4,12);
              \noutreal (a,4,12); outimage;
              \nend
              \nelse if (b*c > 0) then
              \nu2 := (if b<c then b else c)
              \nelse if (b<0.0) then u2 := c
              \nelse u2 := b;
              \nu1 := x1 – u2;
              \nu1 := 1.0\/u1;\u00a0\u00a0 \u00a0u2 := 1.0\/u2;
              \nend;
              \nend;<\/p>\n

              ref(serve) server;
              \nref(arrprocess) arrp;
              \nref(demon) sprocess;<\/p>\n

              q :- new head;
              \nserver :- new serve;
              \narrp\u00a0\u00a0 :- new arrprocess;
              \nsprocess :- new demon;
              \nactivate sprocess after current;
              \nactivate arrp after current;
              \nactivate server after current;
              \nmstime := 8.0;
              \nmitime := 27.3;
              \nV1 := 109827;\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0V2 := 87611;<\/p>\n

              sizeofq := 0;\u00a0\u00a0 \u00a0rtime := 0.0;\u00a0\u00a0 \u00a0qtime := 0.0;\u00a0\u00a0 \u00a0nrsamps := 0;
              \na := 3600 * 24 * 7;\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0comment 1 week;
              \nlastidle := 0.0;\u00a0\u00a0 \u00a0sersum := 0.0;\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0arrtsum := 0.0;
              \nq0samp := 0.0;\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0q1samp := 0.0;\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0sersum := 0.0;
              \nnidle := 0;\u00a0\u00a0 \u00a0narr := 0;\u00a0\u00a0 \u00a0nsers := 0;\u00a0\u00a0 \u00a0nsamps := 0;
              \nq2samp := 0.0;\u00a0\u00a0 \u00a0nzsamps := 0;
              \nls := 0.0;\u00a0\u00a0 \u00a0sizeint := 0.0;\u00a0\u00a0 \u00a0ns := 0;
              \nt1 := 0.0;\u00a0\u00a0 \u00a0t2 := 900.0;
              \nx1 := inreal;\u00a0\u00a0 \u00a0x1 := inreal;\u00a0\u00a0 \u00a0x2 := inreal; x2 := x2*x2;
              \ncomment\u00a0\u00a0 \u00a0outtext (”\u00a0 Mean is “) outreal (x1,4,12)
              \nouttext (” SD is “) outreal(x2,4,12)
              \noutimage;
              \ngetmeans (x1,x2);
              \ncomment\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0outtext (“U1 is “) outreal (1.0\/u1,4,12)
              \nouttext (“U2 is “) outreal (1.0\/u2,4,12)
              \noutimage;<\/p>\n

              hold(a);\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0comment 1 day;<\/p>\n

              outtext (“Single Server Queueing System – Exponential arrival\/service times.”);
              \noutimage;<\/p>\n

              outtext (“Specified interarrival time was: “);
              \noutreal (mitime, 4, 12);
              \nouttext (”\u00a0\u00a0 Measured interarrival time was “);
              \noutreal (arrtsum\/narr, 4, 12);
              \noutimage;
              \nmitime := arrtsum\/narr;<\/p>\n

              outtext (“Specified service time was “);
              \noutreal (mstime, 4, 12);
              \nouttext (”\u00a0\u00a0\u00a0\u00a0 Measured service time was “);
              \noutreal (sersum\/nsers, 4, 12);
              \noutimage;<\/p>\n

              outtext (“Total idle time was “); outreal (idtime,4,12);
              \nouttext (” in “); outint(nidle, 10); outtext (” idle periods.”);
              \noutimage;
              \nouttext(“Mean idle period was “); outreal(idtime\/nidle,5,12);
              \noutimage;
              \nouttext(“Probability of being idle was “); b := idtime\/a;
              \noutreal (b, 5,12); outtext (” Predicted was “); c := mstime\/mitime;
              \noutreal (1.0-c, 5,12); outimage;<\/p>\n

              d := c\/(1.0-c);
              \nouttext (“Mean system size was “); outreal (q0samp\/nsamps, 4, 12);
              \nouttext (” Predicted was “); outreal (d, 4, 12);
              \ne := sizeint\/a;
              \noutreal (e, 4, 12);
              \noutimage;<\/p>\n

              e := 1.0\/(1.0-c);
              \nouttext (“Mean queue size (>0) was “);
              \nif (nzsamps <> 0) then
              \noutreal (q1samp\/nzsamps,4,12);
              \nouttext (” Predicted was “); outreal (e, 4, 12);
              \noutimage;<\/p>\n

              e := (c*c)\/(1.0-c);
              \nouttext (“Mean queue size was “);
              \nif (nsamps <> 0) then
              \noutreal (q2samp\/nsamps,4,12);
              \nouttext (” Predicted was “); outreal (e, 4, 12);
              \noutimage;<\/p>\n

              e := 1.0\/( (1.0\/mstime) – (1.0\/mitime) );
              \nouttext (“Mean response time was “); outreal (rtime\/nrsamps,4,12);
              \nouttext (” Predicted was “); outreal (e, 4, 12);
              \noutimage;<\/p>\n

              a := 1.0\/mitime;\u00a0\u00a0 \u00a0b := 1.0\/mstime;
              \ne := a\/(b*(b-a));
              \nouttext (“Mean queue wait time was “); outreal (qtime\/nrsamps,4,12);
              \nouttext (” Predicted was “); outreal (e, 4, 12);
              \noutimage;<\/p>\n

              end;<\/p>\n

               <\/p>\n

               <\/p>\n","protected":false},"excerpt":{"rendered":"

              \u00a0Chapter 9 Here’s were we will put color versions of the images, updates, and other extras. How do you ensure that the simulation you design simulates your model faithfully? In order to have confidence that the key elements of the simulation are faithful implementations of the original system it is necessary to verify both the […]<\/p>\n","protected":false},"author":1,"featured_media":82,"parent":0,"menu_order":0,"comment_status":"open","ping_status":"closed","template":"","meta":{"_genesis_hide_title":false,"_genesis_hide_breadcrumbs":false,"_genesis_hide_singular_image":false,"_genesis_hide_footer_widgets":false,"_genesis_custom_body_class":"","_genesis_custom_post_class":"","_genesis_layout":"","jetpack_post_was_ever_published":false,"footnotes":""},"class_list":{"0":"post-38","1":"page","2":"type-page","3":"status-publish","4":"has-post-thumbnail","6":"entry"},"jetpack_shortlink":"https:\/\/wp.me\/P4QGQz-C","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"http:\/\/minkhollow.ca\/books\/index.php?rest_route=\/wp\/v2\/pages\/38","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/minkhollow.ca\/books\/index.php?rest_route=\/wp\/v2\/pages"}],"about":[{"href":"http:\/\/minkhollow.ca\/books\/index.php?rest_route=\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"http:\/\/minkhollow.ca\/books\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/minkhollow.ca\/books\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=38"}],"version-history":[{"count":11,"href":"http:\/\/minkhollow.ca\/books\/index.php?rest_route=\/wp\/v2\/pages\/38\/revisions"}],"predecessor-version":[{"id":499,"href":"http:\/\/minkhollow.ca\/books\/index.php?rest_route=\/wp\/v2\/pages\/38\/revisions\/499"}],"wp:featuredmedia":[{"embeddable":true,"href":"http:\/\/minkhollow.ca\/books\/index.php?rest_route=\/wp\/v2\/media\/82"}],"wp:attachment":[{"href":"http:\/\/minkhollow.ca\/books\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=38"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}