7.3 Fast multicore code with new pgcc compiler

Now we will see our first example of new pragmas that are defined in the OpenACC standard.

Same Command line and helper functions as before

#include <stdio.h> #include <unistd.h> #include <stdlib.h> #include <ctype.h> void getArguments(int argc, char *argv[], int * n, int *numThreads); int isNumber(char s[]); void exitWithError(char cmdFlag, char ** argv); void Usage(char *program); void getArguments(int argc, char *argv[], int * n, int *numThreads) { char *nvalue; // number of elements char *numThreads_value; int c; // result from getopt calls while ((c = getopt (argc, argv, "n:t:")) != -1) { switch (c) { case 'n': if (isNumber(optarg)) { nvalue = optarg; *n = atoi(nvalue); } else { exitWithError(c, argv); } break; case 't': if (isNumber(optarg)) { numThreads_value = optarg; *numThreads = atoi(numThreads_value); } else { exitWithError(c, argv); } break; case ':': printf("Missing arg for %c\n", optopt); Usage(argv[0]); exit(EXIT_FAILURE); break; case '?': if (isprint (optopt)) { fprintf (stderr, "Unknown option `-%c'.\n", optopt); Usage(argv[0]); exit(EXIT_FAILURE); } else { fprintf (stderr, "Unknown non-printable option character `\\x%x'.\n", optopt); Usage(argv[0]); exit(EXIT_FAILURE); } break; } } } int isNumber(char s[]) { for (int i = 0; s[i]!= '\0'; i++) { if (isdigit(s[i]) == 0) return 0; } return 1; } void exitWithError(char cmdFlag, char ** argv) { fprintf(stderr, "Option -%c needs a number value\n", cmdFlag); Usage(argv[0]); exit(EXIT_FAILURE); } void Usage(char *program) { fprintf(stderr, "Usage: %s [-n numElements] [-t numThreads]\n", program); }

/* * Functions for setting up arrays and error checking */ #include <stdio.h> #include <math.h> // To reset the arrays for each trial void initialize(float *x, float *y, int N) { // initialize x and y arrays on the host for (int i = 0; i < N; i++) { x[i] = 1.0f; y[i] = 2.0f; } } // showVec prints the vector for debugging void showVec(float *vector, int n) { for (int i = 0; i < n; i++) { printf("%f ", vector[i]); if ((i+1)%10 == 0) printf("\n"); } printf("\n"); } // check whether the kernel functions worked as expected void checkForErrors(float *y, int N) { // Check for errors (all values should be 3.0f) float maxError = 0.0f; float diff = 0.0f; for (int i = 0; i < N; i++) { diff = fabs(y[i]-3.0f); maxError = diff > maxError ? diff : maxError; } printf("Max error: %f\n", maxError); }

The OpenACC multicore code

The pgcc compiler can use OpenACC pragmas to compile a multicore version of the code that runs on the CPU. Note in the function called CPUadd below, the pragma line is slightly different from those we have seen for OpenMP.

Notice how in this case we start with a clause called num_gangs() that is equivalent to using num_threads() in OpenMP pragmas. Most of our OpenMP examples so far used the omp_set_num_threads() function in main instead.

As you try running this code, We describe below how you can eliminate the use of the num_gangs() clause and simply let the compiler use the most cores available on the machine you have compiled the code on.

Note that we still use the OpenMP function omp_get_wtime() to help use time the code.

Note the compiler arguments shown below the code. We expose these so that you can see what is needed to compile a multicore version using pgcc. In particular these two new compiler flags:

• -acc=multicore : signifies that code should be built to use multiple cores and therefore multiple threads, as indicated before blocks with #pragma acc parallel

• -Minfo=opt : this enables additional information to be reported by the compiler about how the code was optimized.

#include <math.h> // fmax function #include <stdio.h> // printf #include <stdlib.h> // malloc #include <omp.h> // openMP (just for timing) #define PRINT_LIMIT 40 // CPU version of add with acc pragma // Note the use of num_gangs() to indicate the number of threads // can slow down the code. void CPUadd(int n, float *x, float *y, int numThreads) { #pragma acc parallel loop num_gangs(numThreads) // A. comment // #pragma acc parallel loop // B. uncomment, numThreads setting ignored for (int i = 0; i < n; i++) { y[i] = x[i] + y[i]; } } int main(int argc, char **argv) { printf("Vector addition using acc pragma pgcc compiler.\n"); // Set up size of arrays for vectors // large N int N = 1024*1024*1024; int numThreads =1; // get command line args to change size of array getArguments(argc, argv, &N, &numThreads); printf("size (N) of 1D arrays are: %d\n\n", N); // host vectors float *x, *y; // Size, in bytes, of each vector size_t bytes = N*sizeof(float); // Allocate memory for each vector on host x = (float*)malloc(bytes); y = (float*)malloc(bytes); // initialize x and y arrays on the host initialize(x, y, N); // set values in each vector if (N <= PRINT_LIMIT) { // debug printf("x:\n"); showVec(x, N); printf("y:\n"); showVec(y, N); } printf("add vectors on host using %d threads\n", numThreads); double start, end; start = omp_get_wtime(); CPUadd(N, x, y, numThreads); end = omp_get_wtime(); printf("Time: %lf secs\n", end - start); if (N <= PRINT_LIMIT) { // debug printf("y result:\n"); showVec(y, N); } checkForErrors(y, N); printf("execution complete\n"); // Release host memory free(x); free(y); return 0; }

Added compiler information

As with prior examples of using pgcc, after the program output, there is a line that looks like this: ===== STANDARD ERROR =====. What follows this is the output from the pgcc compiler. We indicated that we wanted this output by using the compiler flag ‘-Minfo=opt’. This pgcc compiler provides this option so that you can see the optimizations the compiler used as a result of including the -fast option.

Exercises

• Try varying the number of threads: Remove the ‘-n’, ‘10’ from the command line arguments and try each of these: [‘-t’, ‘2’] and [‘-t’, ‘4’] and [‘-t’, ‘8’] to see how you gain some improvement in the running time.

• What do you notice about the running times of this version versus the two in the previous section?

• Change the code to let the compiler chose the maximum number of threads: In the CPUadd() function, look for the pragma line, which is labeled A. in the comment behind it. Comment out this entire pragma and uncomment the line below it with a slightly different pragma, labeled B. in the comment behind it. In this case, the compiler will ignore any thread settings you indicate on the command line and use the number of cores on the machine it is compiled on.

Which is best?

We once again see that it pays to experiment with different compilers and find out what works best for your application. For pgcc and OpenACC compilers, the decision of the compiler writers is to use the most threads that can be run on available cores on your machine. In some cases this turns out to be the best way to use their multicore version, since if you move the code from one machine to another and recompile, it can take advantage of what you have available. You often simply want to do this for your application and get the most out of the resources you have. You have to be careful, though, and make sure that your performance isn’t degrading by using too many threads.

You have attempted of activities on this page