{"id":42,"date":"2026-08-06T09:31:35","date_gmt":"2026-08-06T12:31:35","guid":{"rendered":"http:\/\/blog.pej-mps.org\/?p=42"},"modified":"2026-08-06T09:31:35","modified_gmt":"2026-08-06T12:31:35","slug":"a-statistical-assessment-of-our-2026-2027-proic-results","status":"publish","type":"post","link":"http:\/\/blog.pej-mps.org\/?p=42","title":{"rendered":"A Statistical Assessment of Our 2026\u20132027 ProIC Results"},"content":{"rendered":"\n

Overview<\/h1>\n\n\n\n

The spreadsheet finally resolved the extraction problems posed by the original PDF and made it possible to analyse the results systematically.<\/p>\n\n\n\n

The first comparison involved a selected group of twelve supervisors with varying numbers of submitted work plans. For each supervisor, the arithmetic mean of all registered plans was calculated.<\/p>\n\n\n\n

Rank<\/th>Supervisor<\/th>Plans<\/th>Mean score<\/th><\/tr><\/thead>
1<\/td>A.G.M.A.<\/td>1<\/td>99.3243<\/td><\/tr>
2<\/td>M.F.P.C.C.<\/td>4<\/td>96.0321<\/td><\/tr>
3<\/td>L.D.R.<\/td>4<\/td>95.7925<\/td><\/tr>
4<\/td>D.B.A.F.<\/td>1<\/td>95.6667<\/td><\/tr>
5<\/td>A.C.H.<\/td>4<\/td>94.8388<\/td><\/tr>
6<\/td>T.J.C.A.<\/td>2<\/td>94.3702<\/td><\/tr>
7<\/td>A.F.M.P.<\/td>2<\/td>91.1653<\/td><\/tr>
8<\/strong><\/td>V.C.R.A.D.<\/strong><\/td>5<\/strong><\/td>90.5515<\/strong><\/td><\/tr>
9<\/td>L.M.C.I.<\/td>1<\/td>88.0726<\/td><\/tr>
10<\/td>C.S.C.<\/td>1<\/td>84.8750<\/td><\/tr>
11<\/td>V.B.M.P.<\/td>3<\/td>81.6244<\/td><\/tr>
12<\/td>L.P.F.N.<\/td>2<\/td>78.1427<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n

The methodological problem<\/h2>\n\n\n\n

A direct comparison of averages becomes precarious when some supervisors submitted only one plan and others submitted five. A single outstanding result can produce a very high average, whereas a supervisor with several plans is being assessed across a much broader body of work.<\/p>\n\n\n\n

V.C.R.A.D. submitted five plans, all of which received scores between 90.14 and 91.01. The dispersion was exceptionally small:<\/p>\n\n\n\n

Measure<\/th>Result<\/th><\/tr><\/thead>
Lowest score<\/td>90.1375<\/td><\/tr>
Highest score<\/td>91.0068<\/td><\/tr>
Range<\/td>0.8693<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n

The five individual scores were:<\/p>\n\n\n\n

90.1375 \u00b7 90.3209 \u00b7 90.3857 \u00b7 90.9066 \u00b7 91.0068<\/strong><\/p>\n\n\n\n

This degree of uniformity suggests that different evaluators, assessing different plans, arrived at almost exactly the same judgement concerning their overall quality. From a statistical perspective, that consistency is at least as significant as the average itself.<\/p>\n\n\n\n

When the comparison is restricted to supervisors with four or five plans\u2014thus creating a group with a more comparable supervisory workload\u2014the result is:<\/p>\n\n\n\n

Supervisor<\/th>Plans<\/th>Mean score<\/th><\/tr><\/thead>
M.F.P.C.C.<\/td>4<\/td>96.03<\/td><\/tr>
L.D.R.<\/td>4<\/td>95.79<\/td><\/tr>
A.C.H.<\/td>4<\/td>94.84<\/td><\/tr>
V.C.R.A.D.<\/strong><\/td>5<\/strong><\/td>90.55<\/strong><\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n

Within this smaller group, V.C.R.A.D. ranks fourth. The comparison is methodologically more meaningful than one that places a supervisor with five plans beside someone represented by a single score.<\/p>\n\n\n\n

The results are not among the extreme peaks of 96\u201399, but they are unusually stable and comfortably above 90. In a call whose scoring distribution was relatively generous, this regularity remains a positive and noteworthy result.<\/p>\n\n\n\n

Position within the selected group<\/h2>\n\n\n\n

Using the arithmetic mean alone, without weighting for the number of plans, V.C.R.A.D. ranked eighth among the twelve selected supervisors<\/strong>:<\/p>\n\n\n\n

    \n
  • seven had higher averages;<\/li>\n\n\n\n
  • four had lower averages.<\/li>\n<\/ul>\n\n\n\n

    This selected group was not a random sample of the university. It consisted largely of colleagues with substantial experience in research and supervision. Eighth place within such a group therefore does not necessarily imply a modest result in relation to the complete population of supervisors.<\/p>\n\n\n\n

    The central feature remains consistency: none of the five scores fell below 90 or rose above 91. The five projects were judged within an exceptionally narrow band.<\/p>\n\n\n\n

    Comparison with all work plans<\/h2>\n\n\n\n

    The complete dataset contained:<\/p>\n\n\n\n

    Measure<\/th>Result<\/th><\/tr><\/thead>
    Total number of plans<\/td>2,169<\/td><\/tr>
    Overall mean score<\/td>89.03<\/td><\/tr>
    Overall median score<\/td>92.60<\/td><\/tr>
    Mean of the five plans considered here<\/td>90.5515<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n

    The five-plan average was therefore 1.52 points above the overall mean<\/strong>, although it remained below the overall median.<\/p>\n\n\n\n

    The fact that the general mean was substantially lower than the median indicates a negatively skewed distribution: a number of low-scoring plans pulled the mean down, while a large share of the approved projects remained concentrated in the higher ranges.<\/p>\n\n\n\n

    A direct comparison between the five-plan average and the 2,169 individual plan scores placed it at approximately the 42nd percentile<\/strong>. This comparison is useful as a broad indication, but it is not methodologically ideal, since it compares the average of five plans with individual project scores.<\/p>\n\n\n\n

    A fairer comparison requires calculating the average for each supervisor and then ranking supervisors rather than individual plans.<\/p>\n\n\n\n

    Position among all supervisors<\/h2>\n\n\n\n

    After grouping the data by supervisor registration code, the complete population consisted of 1,049 supervisors<\/strong>.<\/p>\n\n\n\n

    V.C.R.A.D.\u2019s average of 90.5515<\/strong> ranked:<\/p>\n\n\n\n

    634th out of 1,049 supervisors.<\/strong><\/p>\n\n\n\n

    In other words:<\/p>\n\n\n\n

      \n
    • approximately 39.7% of supervisors had lower averages;<\/li>\n\n\n\n
    • approximately 60.3% had higher averages.<\/li>\n<\/ul>\n\n\n\n

      This absolute ranking answers the most literal version of the question, but it has an obvious limitation: many supervisors were represented by only one plan. For them, a single exceptional\u2014or weak\u2014project determined the entire average. An average based on five projects is considerably more stable.<\/p>\n\n\n\n

      Comparisons by minimum number of plans<\/h2>\n\n\n\n

      To reduce the effect of very small samples, the ranking was recalculated among supervisors with progressively larger numbers of submitted plans.<\/p>\n\n\n\n

      Minimum number of plans<\/th>Supervisors included<\/th>Position<\/th><\/tr><\/thead>
      1<\/td>1,049<\/td>634th<\/td><\/tr>
      2<\/td>601<\/td>367th<\/td><\/tr>
      3<\/td>276<\/td>155th<\/td><\/tr>
      4<\/td>136<\/td>76th<\/td><\/tr>
      5<\/td>65<\/td>32nd<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n

      The final comparison is the most informative.<\/p>\n\n\n\n

      Among the 65 supervisors who submitted five or more plans<\/strong>, V.C.R.A.D. ranked:<\/p>\n\n\n\n

      32nd out of 65.<\/strong><\/p>\n\n\n\n

      This places the result almost exactly at the centre of the distribution and marginally within the upper half.<\/p>\n\n\n\n

      Quartile position<\/h2>\n\n\n\n

      Among these 65 high-volume supervisors, the quartiles can be represented approximately as follows:<\/p>\n\n\n\n

        \n
      • first quartile: positions 1\u201316;<\/li>\n\n\n\n
      • second quartile: positions 17\u201332;<\/li>\n\n\n\n
      • third quartile: positions 33\u201348;<\/li>\n\n\n\n
      • fourth quartile: positions 49\u201365.<\/li>\n<\/ul>\n\n\n\n

        At 32nd place, V.C.R.A.D. stands at the lower boundary of the second quartile\u2014or, stated more plainly, just inside the upper half of the group.<\/p>\n\n\n\n

        The broad distribution suggests three approximate performance bands:<\/p>\n\n\n\n

          \n
        • 90\u201391:<\/strong> the largest concentration;<\/li>\n\n\n\n
        • 93\u201395:<\/strong> a clearly stronger upper group;<\/li>\n\n\n\n
        • 96\u201399:<\/strong> the statistical elite of the call.<\/li>\n<\/ul>\n\n\n\n

          To move into the top twenty-five among supervisors with at least five plans would have required an average close to 92.5\u201393. An average near 94 would have been necessary for the top twenty, and approximately 95.5\u201396 for the top ten.<\/p>\n\n\n\n

          These are approximate thresholds, but they reflect the general shape of the observed distribution.<\/p>\n\n\n\n

          A refined performance index<\/h2>\n\n\n\n

          A simple average does not account for the typical result, the consistency of the scores, or the number of plans submitted. To address these limitations without producing an artificially flattering ranking, a deliberately conservative composite index was adopted:Index<\/mtext>=<\/mo>0.7<\/mn>\u00d7<\/mo>Median<\/mtext>+<\/mo>0.3<\/mn>\u00d7<\/mo>Mean<\/mtext>\u2212<\/mo>0.5<\/mn>\u00d7<\/mo>Standard Deviation<\/mtext>+<\/mo>Volume Bonus<\/mtext><\/mrow>\\text{Index} = 0.7\\times\\text{Median} + 0.3\\times\\text{Mean} – 0.5\\times\\text{Standard Deviation} + \\text{Volume Bonus}<\/annotation><\/semantics><\/math>Index=0.7\u00d7Median+0.3\u00d7Mean\u22120.5\u00d7Standard Deviation+Volume Bonus<\/p>\n\n\n\n

          The volume bonus was set at 0.15 points for each additional plan after the first<\/strong>, capped at six plans. The maximum possible bonus was therefore 0.75 points. This acknowledges the greater evidential value of multiple submissions without allowing volume to dominate the score.<\/p>\n\n\n\n

          For V.C.R.A.D., the calculation was:<\/p>\n\n\n\n

          Component<\/th>Value<\/th><\/tr><\/thead>
          Mean<\/td>90.5515<\/td><\/tr>
          Median<\/td>90.3857<\/td><\/tr>
          Standard deviation<\/td>0.3422<\/td><\/tr>
          Dispersion penalty<\/td>\u22120.1711<\/td><\/tr>
          Volume bonus for five plans<\/td>+0.6000<\/td><\/tr>
          Final index<\/strong><\/td>90.8643<\/strong><\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n

          Thus:0.7<\/mn>(<\/mo>90.3857<\/mn>)<\/mo>+<\/mo>0.3<\/mn>(<\/mo>90.5515<\/mn>)<\/mo>\u2212<\/mo>0.5<\/mn>(<\/mo>0.3422<\/mn>)<\/mo>+<\/mo>0.60<\/mn>=<\/mo>90.8643<\/mn><\/mrow>0.7(90.3857) + 0.3(90.5515) – 0.5(0.3422) + 0.60 = 90.8643<\/annotation><\/semantics><\/math>0.7(90.3857)+0.3(90.5515)\u22120.5(0.3422)+0.60=90.8643<\/p>\n\n\n\n

          Under this refined measure, the ranking changed as follows:<\/p>\n\n\n\n

          Comparison<\/th>Simple average<\/th>Refined index<\/th><\/tr><\/thead>
          All 1,049 supervisors<\/td>634th<\/td>606th<\/strong><\/td><\/tr>
          Supervisors with five or more plans<\/td>32nd<\/td>30th<\/strong><\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n

          The refined index therefore raises the position by twenty-eight places in the complete population and by two places among supervisors with at least five plans.<\/p>\n\n\n\n

          This is a modest improvement, as it should be. The measure recognises the value of consistency and a broader supervisory workload, but it does not erase genuine differences of four, five, or eight points in average scores. A stable set of results around 90 should not automatically outrank a similarly stable set around 96.<\/p>\n\n\n\n

          Tests using slightly different penalties for dispersion produced only limited changes. The general position varied between approximately 582nd and 619th among all supervisors, and between 29th and 32nd among those with five or more plans. The conclusion is therefore reasonably robust.<\/p>\n\n\n\n

          Final assessment<\/h2>\n\n\n\n

          The most significant feature of the results is not the absolute ranking. It is the combination of:<\/p>\n\n\n\n

            \n
          • five submitted plans;<\/li>\n\n\n\n
          • five scores above 90;<\/li>\n\n\n\n
          • a difference of less than one point between the highest and lowest result;<\/li>\n\n\n\n
          • a standard deviation of only 0.34.<\/li>\n<\/ul>\n\n\n\n

            The results do not place the supervisor within the statistical elite of the call. Nor should a responsible analysis pretend otherwise. They do, however, demonstrate that a new project, involving new students and a highly specialised field with little institutional precedent in Brazil, was regarded as solid across the board.<\/p>\n\n\n\n

            The result is therefore best described as very satisfactory rather than spectacular<\/strong>: approximately central among supervisors with a comparable volume of work, but distinguished by exceptional internal consistency.<\/p>\n\n\n\n

            In peer-review processes, such consistency is often more informative than an isolated peak. It indicates that the strength of the submission did not depend on a single exceptional project. Instead, the entire group of work plans was assessed at essentially the same, consistently high level.<\/p>\n","protected":false},"excerpt":{"rendered":"

            Overview The spreadsheet finally resolved the extraction problems posed by the original PDF and made it possible to analyse the results systematically. The first comparison involved a selected group of twelve supervisors with varying numbers of submitted work plans. For each supervisor, the arithmetic mean of all registered plans was calculated. Rank Supervisor Plans Mean […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-42","post","type-post","status-publish","format-standard","hentry","category-general"],"_links":{"self":[{"href":"http:\/\/blog.pej-mps.org\/index.php?rest_route=\/wp\/v2\/posts\/42","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/blog.pej-mps.org\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/blog.pej-mps.org\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/blog.pej-mps.org\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/blog.pej-mps.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=42"}],"version-history":[{"count":1,"href":"http:\/\/blog.pej-mps.org\/index.php?rest_route=\/wp\/v2\/posts\/42\/revisions"}],"predecessor-version":[{"id":43,"href":"http:\/\/blog.pej-mps.org\/index.php?rest_route=\/wp\/v2\/posts\/42\/revisions\/43"}],"wp:attachment":[{"href":"http:\/\/blog.pej-mps.org\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=42"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/blog.pej-mps.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=42"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/blog.pej-mps.org\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=42"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}