Montañez
Show Menu
nuxt

Using gRPC (CommonJS) with Nuxt 3

Joseph Montañez••
#howto #nuxt #typescript #esm #commonjs

Table of Contents

The issue at hand, grpc_tools_node_protoc does not yet support ESM compiling of protocol buffers or gRPC clients. Instead, you're stuck with CommonJS. This isn't a problem as you can use CommonJS in ESM albeit, its not great. However, Nuxt 3's TypeScript is setup with ESNext for the module section and you cannot just have .js files as those are assumed to be ESM files, they need to be renamed to .cjs to allow CommonJS importing. You can switch to CommonJS for the entire Nuxt 3 project, but I didn't want to do that.

Please before you jump down this rabbit hole give grpc-web and Envoy Proxy a try, they are alternatives. I avoided going that route as I didn't want to stand up yet another service, so it's just a warning, the following contains dragons...

One last note! This only works on the server side of Nuxt. This is because gRPC by nature is not compatible with any web browser to date. So if you really want to work directly on the client side only with gRPC then you'll need to use gRPC-web.

Converting .proto to NodeJS and TypeScript

First lets install the various packages:

npm i @grpc/grpc-js --save
npm i @grpc/proto-loader --save
npm i protoc-gen-js --save-dev
npm i ts-protoc-gen --save-dev
npm i grpc_tools_node_protoc_ts --save-dev

Now convert them into the .js and .d.ts files:

grpc_tools_node_protoc \
    --js_out=import_style=commonjs,binary:./generated \
    --grpc_out=grpc_js:./generated \
    --ts_out=grpc_js:./generated \
    --plugin=protoc-gen-grpc=`which grpc_tools_node_protoc_plugin` \
    --plugin=protoc-gen-ts=`which protoc-gen-ts` \
    -I ./../protos \
    ./../protos/authentication.proto \
    ./../protos/account.proto \
    ./../protos/job_posting.proto

Using Generated Proto And gRPC Client Code

Now do some basic gRPC'ing...

import { credentials } from "@grpc/grpc-js";
import {AuthServiceClient} from "~/generated/authentication_grpc_pb";
import {
    LogInRequest,
    LogInResponse
} from '~/generated/authentication_pb';

const client = new AuthServiceClient(
    process.env.GRPC_ENDPOINT ?? "http://localhost:8000",
    process.env.GRPC_SECURE === 'true' ? credentials.createSsl() : credentials.createInsecure()
);

new Promise((resolve, reject) => {
    let request = new LogInRequest();
    request.setEmail(email);
    request.setPassword(password);

    client.logIn(request, (error: ServiceError | null, response: LogInResponse) => {
        if (error) {
            console.error('Error logging in:', error.message);
            reject(error);
        } else {
            console.log('Login successful:', response);
            resolve(response);
        }
    });
});

That's it! have a great day!.... except!

  import { AuthServiceClient } from 'generated/authentication_grpc_pb.js';
  ^^^^^^^^^^^^^^^^^
  SyntaxError: The requested module 'generated/authentication_grpc_pb.js' does not provide an export named 'AuthServiceClient'
  at ModuleJob._instantiate (node:internal/modules/esm/module_job:134:21)
  at async ModuleJob.run (node:internal/modules/esm/module_job:217:5)
  at async ModuleLoader.import (node:internal/modules/esm/loader:323:24)
  at async loadESM (node:internal/process/esm_loader:28:7)
  at async handleMainPromise (node:internal/modules/run_main:113:12)

Now comes hell... I've tried CommonJS to ESM via Babel and cjstoesm, neither could fix the issue... Instead I gave up on converting to ESM and instead stuck with CommonJS, but two minor fixes.

  • Files generated in .js must be renamed to .cjs
  • Imports (require) must import .cjs rather than .js

This here is a simple NodeJS script to do just that:

convert.js

import fs from 'fs/promises';
import path from 'path';
import { fileURLToPath } from 'url';

const dirname = path.dirname(fileURLToPath(import.meta.url));
const generatedPath = path.join(dirname, 'generated');

async function renameFiles() {
    try {
        // Recursively get all files in the directory
        const files = await getFiles(generatedPath);

        // Filter and rename .js files to .cjs
        for (const file of files) {
            if (file.endsWith('.js')) {
                const newFile = `${file.slice(0, -3)}.cjs`;
                await fs.rename(file, newFile);
                console.log(`Renamed: ${file} to ${newFile}`);
            }
        }

        // Get all files again after renaming
        const updatedFiles = await getFiles(generatedPath);

        // Replace .js to .cjs in file contents
        for (const file of updatedFiles) {
            if (file.endsWith('.cjs')) {
                let content = await fs.readFile(file, 'utf8');
                content = content.replace(/\.js(["'`,; ])/g, '.cjs$1');
                await fs.writeFile(file, content);
                console.log(`Updated file contents of: ${file}`);
            }
        }
    } catch (error) {
        console.error('Error processing files:', error);
    }
}

async function getFiles(dir) {
    let files = [];
    const items = await fs.readdir(dir, { withFileTypes: true });
    for (const item of items) {
        const resPath = path.resolve(dir, item.name);
        if (item.isDirectory()) {
            files = files.concat(await getFiles(resPath));
        } else {
            files.push(resPath);
        }
    }
    return files;
}

renameFiles().then(r => {});

So that's it right? Nope, we are not there yet. Using CommonJS requires async loading of modules, which means import still will not work

Wrapping CommonJS Modules With Async Loading

Let's take the first part, setting up the gRPC client:

import { credentials } from "@grpc/grpc-js";
import {AuthServiceClient} from "~/generated/authentication_grpc_pb";

const client = new AuthServiceClient(
    process.env.GRPC_ENDPOINT ?? "http://localhost:8000",
    process.env.GRPC_SECURE === 'true' ? credentials.createSsl() : credentials.createInsecure()
);

We need to refactor this into an async load for example:

server/libs/authServiceClient.ts

import { credentials } from "@grpc/grpc-js";
import { AuthServiceClient as AuthServiceClientType } from '~/generated/authentication_grpc_pb';

let AuthClient: AuthServiceClientType | null = null;

const authServiceClientPromise = import('~/generated/authentication_grpc_pb')
    .then(module => {
        AuthClient = new module.AuthServiceClient(
            process.env.GRPC_ENDPOINT ?? "http://localhost:8000",
            process.env.GRPC_SECURE === 'true' ? credentials.createSsl() : credentials.createInsecure()
        );
        return AuthClient;  // Return the initialized client
    })
    .catch(error => {
        console.error('Failed to load AuthServiceClient:', error);
        throw error;
    });

export { AuthClient, authServiceClientPromise };

Next we need to wrap the responses and requests inside the protocol buffer we converted.

server/libs/authTypes.ts

const authTypesPromise = import('~/generated/authentication_pb')
    .then(module => ({
        ForgotPasswordRequest: module.default.ForgotPasswordRequest,
        ForgotPasswordResponse: module.default.ForgotPasswordResponse,
        LogInRequest: module.default.LogInRequest,
        LogInResponse: module.default.LogInResponse,
        SignUpRequest: module.default.SignUpRequest,
        SignUpResponse: module.default.SignUpResponse,
        ValidateTokenRequest: module.default.ValidateTokenRequest,
        ValidateTokenResponse: module.default.ValidateTokenResponse
    }))
    .catch(error => {
        console.error('Failed to load authentication types:', error);
        throw error;
    });

export default authTypesPromise;

Using The Wrapped CommonJS Protocol Buffer / gRPC

Now for the final step is to integrate the wrapped CommonJS into something we can await / async! Note, we can use import type, of say LogInResponse as that comes from the .d.ts file. But when we need to initialize an object like LogInRequest, then we have the load it via CommonJS async loading.

server/libs/authService.ts

import {credentials, ServiceError} from "@grpc/grpc-js"
import {AuthClient, authServiceClientPromise} from './authServiceClient';
import authTypesPromise from './authTypes';
import type {LogInResponse} from '~/generated/authentication_pb';

export async function logIn(email: string, password: string): Promise<ServiceError | LogInResponse> {
    await authServiceClientPromise;
    if (!AuthClient) throw new Error("AuthServiceClient not initialized");
    const {
        LogInRequest
    } = await authTypesPromise;

    return new Promise((resolve, reject) => {
        let request = new LogInRequest();
        request.setEmail(email);
        request.setPassword(password);

        AuthClient.logIn(request, (error: ServiceError | null, response: LogInResponse) => {
            if (error) {
                console.error('Error logging in:', error.message);
                reject(error);
            } else {
                console.log('Login successful:', response);
                resolve(response);
            }
        });
    });
}

Then finally pass in a backend call into the gRPC method. This time note I do not import type of LogInResponse. This is because I am using instanceof which is a runtime call and requires the LogInResponse JavaScript class, not the TypeScript type.

server/api/auth/login.post.ts

// @ts-ignore
import { defineEventHandler, H3Event, readBody } from "h3";
import { ServiceError } from "@grpc/grpc-js";
import { logIn } from "~/server/libs/authService";
import authTypesPromise from "~/server/libs/authTypes";

export default defineEventHandler(async (event: H3Event) => {
    const body = await readBody(event);
    const email = body.email;
    const password = body.password;

    try {
        const { LogInResponse } = await authTypesPromise;
        const response = await logIn(email, password);

        if (response instanceof LogInResponse && response.getSuccess()) {
            const user = {
                id: response.getUserid(),
                fullName: response.getFullname(),
                email: response.getEmail(),
                role: 'user',
            };

            return {
                success: true,
                token: response.getToken(),
                user: {
                    id: user.id,
                    role: user.role
                }
            };
        } else {
            return {
                success: false,
                message: (response as ServiceError).message || "Authentication failed"
            };
        }
    } catch (e: any) {
        return {
            success: false,
            message: 'ERROR: ' + e.message
        };
    }
});

Experimenting with TypeScript Guards

There is a problem calling response.getUserid() even after checking the instance type. This is due to TypeScript's inferencing not being strong enough. A quick work around is force casting (response as LogInResponseType).getUserid().
I also tried using a type guard but this still did not work. If you are interested you can setup nitro with top-level async support and implement a type guard.

First we need to declare it with ES2020 or higher nuxt.config.ts

export default defineNuxtConfig({
    //...
    nitro: {
        esbuild: {
            options: {
                target: 'esnext'
            }
        }
    },
})

Now we can set up our type guard:

import {logIn} from "~/server/libs/authService";
import authTypesPromise from "~/server/libs/authTypes";
// @ts-ignore
import  {type LogInResponse as LogInResponseType} from "~/generated/authentication_pb";

const { LogInResponse } = await authTypesPromise;

//-- This is our type guard
function isLogInResponse(response: any): response is LogInResponseType {
    // Here, check for a unique, non-nullable property that only LogInResponse would have
    return response instanceof LogInResponse;
}

export default defineEventHandler(async (event: H3Event) => {
    const body = await readBody(event);
    const email = body.email;
    const password = body.password;

    try {
        const response = await logIn(email, password);

        // We use the type guard here
        if (isLogInResponse(response)) {
            const user = {
                id: (response as LogInResponseType).getUserid(), // We still have to cast anyways =_= due to failed type inferencing
                fullName: (response as LogInResponseType).getFullname(),
                email: (response as LogInResponseType).getEmail(),
                role: 'user',
            };
            //...
        }
    } catch (e) {}
})
← Back to Articles