My Nodejs interview was Disastrous Because I Lacked this Basic Knowledge

This article shall enlighten you on node concurrency, event loop, pros and cons of single and multithreaded applications and frameworks

My Nodejs interview was Disastrous Because I Lacked this Basic Knowledge

Nodejs is a single-threaded event loop-based architecture!!

You : Pssh!! Everyone knows that! This article is another clickbait damn it!
Me : HOLD RIGHT THERE!! If you know that much, you better tell me how node handles 10000 requests concurrently or why is it more popular than multi-threaded frameworks or a multitude of other questions I have prepared to make sure you walk out of this article as a true Nodejs developer!

I’ll make sure to answer all of these questions, so grab a coffee and let the man speak(or in this case, write). I do this because I don’t want you getting roasted like I was in my first bachelor's interview(No, not the TV show! but I do find Tayshia Adams quite hot and Hannah Godwins quite adorable). *Cough*..*Cough* back to the topic………….

Before you put any more time into reading this article, let’s first make sure we are on the same page. Read the following questions…

  1. Can you tell me if Nodejs is Synchronous or Asynchronous and if it is Single or Multi-Threaded?
  2. Can you explain how Nodejs handles multiple requests concurrently despite being Single-threaded?
  3. Can you describe what is an Event Loop?

and the trickiest of them all…….

4. Is Nodejs a language or a framework?

If all or most of the questions above make your brain go….well KABOOM!!

Then I just saved you from a heap of embarrassment when you find yourself in a situation where you just told them you are a Nodejs developer but are clueless when questioned on these basics.

*Sigh* …I know at this point for covering your ass from this hypothetical but a very probable situation, you want to shower me with love, care, and a drink, but to keep covering your behind from these kinds of situations, I need to write more and for that, I need some coffee for which I am broke. So, if you feel extra generous today, consider buying me a coffee here.

Now, I hope the Lord had mercy and led you here before meeting your executioner, so without further ado, let’s start with

The Crucial Basics of Nodejs That You Never Bothered About

Let’s first address the pink elephant in the room. Do you absolutely need to know the following information to build Nodejs applications using any one of the fancy frameworks currently trending…the answer is a spectacular NO!…….BUT! You absolutely need this information to:

  1. Get a job as a Node Developer
  2. Avoid embarrassment when a junior asks you basic Nodejs questions after learning that you are a Node developer.
  3. Building Optimized and Robust Nodejs applications. (Elaborated later on like Writing blocking code vs non-blocking code, efficiently using working threads, and so on)

Now, since that is out of the way, let’s start with a reminder of

What is Nodejs?

Note: Pay close attention here as I explain the answer to the trickiest question from the above

Wikipedia says:

Node.js is an open-source, cross-platform, back-end JavaScript runtime environment that runs on the V8 engine and executes JavaScript code outside a web browser

There you go folks, Nodejs is a runtime environment and since that answers your questions, let’s move on……

HOLD UP!! HOLD UP!! I apologize!! Let me explain!!! No chill, huh.

Usually, languages are of two types: Programming and Scripting. Programming languages need a compiler to compile the code whereas scripting languages without compiling execute the code in runtime. In other words, no matter if it is a programming or scripting language, we need a compiler or interpreter to execute a program.

Chrome’s V8 often called an engine, is a package deal containing an interpreter, compiler, runtime implementation compatible with the runtime interface demanded by the ECMA standard JavaScript conventions.

So Nodejs is not a framework(or a language) but a JavaScript runtime built on Chrome’s V8 JavaScript engine. It extends the power of handling file and network I/O with a preference for asynchronous patterns after including everything that is Javascript.

In simple words, Node.js expands the JavaScript language by providing a larger set of runtime environment primitives — those which are outside the scope of ECMA’s standards. These include things like file handling, network I/O, etc. Javascript does not come equipped with these because for security reasons, javascript originally did not include file I/O for use in the browser. And, it did not need to do networking tasks, because the browser does them.

Is Nodejs Single-threaded and is it Synchronous or Asynchronous?

This question would come up when you learn about how the event loop and how other internal cogs work. They look like they work synchronously but when you dig deeper, you learn the beautiful intricacies of Nodejs. But first, to answer the question, Nodejs is Single-threaded and Asynchronous.

I know this statement begs the question if Node is single-threaded, why is it hailed as the messiah of backend if it can serve only one user at a time….my answer is PATIENCE BOY!! (God of War Style)

Event Loop

The event loop is the answer to all the questions that you never asked for, it is like the KFC secret recipe where everyone knows the ingredients but only a select few know how they are mixed and it is high time you join the upper ranks(not the KFC ofc, but if ya know, you NEED TO let me know).

JS executes all operations on a single thread, but using a few smart data structures, gives us the illusion of multi-threading.

There is an event queue that uses a queue data structure that is responsible for sending new functions to the track for processing.

Whenever an async function is called, it is sent to a browser API. These are APIs built into the browser. Based on the command received from the call stack, the API starts its own single-threaded operation.

You can take any number of examples like the setTimeout method, Axios methods, or any other method that run asynchronously. let’s take the setTimeout operation and look into its lifecycle. When a setTimeoutis called, it is processed in the stack and then sent to the corresponding API which waits till the specified time to send this operation back in for processing.

Where does it send the operation? The event queue. Hence, we have a cyclic system for running async operations in JavaScript. The language itself is single-threaded, but the browser APIs act as separate threads.

The event loop facilitates this process; it constantly checks whether or not the call stack is empty. If it is empty, new functions are added from the event queue. If it is not, then the current function call is processed.

Now, that you understand the event loop, we can dive into understanding how despite being single-threaded, Nodejs behaves like a multi-threaded language. An application of learning this concept is answering the question you have been longing for the entire duration of this article “How does Nodejs handle multiple requests concurrently despite being Single-threaded?”

The answer to this is that there is one process, multiple threads, but only one thread dedicated to parsing and executing javascript code. the other threads are started from C++ bindings called from the js.

To better understand, generally web apps do this:

user does an action
       │
       v
 the application starts processing action
   └──> make read or write database request
          └──> backend does nothing until request completes
 request complete
   └──> database returns the result
          └──> backend sends result to user

As you can see, a lot of time the backend itself does nothing and so CPU utilization is 0%

Multithreaded network app:

And multi-threaded web apps handle the same workload by basically behaving like the second generation of a very rich parent but instead of money, they spend memory.

request 1 ──> spawn thread(Takes memory say memory1)
              └──> wait for database request
                     └──> answer request(Release that memory1)
request 2 ──> spawn thread(Takes memory say memory2)
              └──> wait for database request
                     └──> answer request(Release that memory2)
request 3 ──> spawn thread(Takes memory say memory3)
              └──> wait for database request
                     └──> answer request(Release that memory3)

So each thread again spends most of its time using 0% CPU waiting for the database to return data. While doing so they have had to allocate the memory and RAM required for a thread which includes a completely separate program stack for each thread etc. Also, they would have to start a thread which while is not as expensive as starting a full process is still not exactly cheap.

Single-threaded event loop

Since we spend most of our time we let the CPU sit idle at 0%, why not use it more efficiently by running some code while it is idle? That way, each request will still get the same amount of CPU time as multithreaded applications but we don’t need to start a thread and so this is how Single-threaded Event loop work:

request ──> make database request
request ──> make database request
request ──> make database request
database request complete ──> send response
database request complete ──> send response
database request complete ──> send response

Fails of single-thread approach

A single-threaded app fails big if you need to do lots of CPU calculations before returning the data. Now, I don’t mean a for loop processing the database result. That’s still mostly O(n). What I mean is things like doing Fourier transform (mp3 encoding for example), ray tracing (3D rendering), etc.

Another pitfall of single-threaded apps is that they will only utilize a single CPU core. So if you have a quad-core server (not uncommon nowadays) you’re not using the other 3 cores.

Fails of Multi-thread approach

A multithreaded app fails big time when handling multiple requests concurrently because it allocates lots of RAM per thread, and this huge amount of RAM stays idle while the job is being done like for example when you call a query to your database until the database returns the thread and the RAM are going nowhere thus, in essence, blocking out your RAM for no reason which in other words means we can potentially end up being slower than single-threaded apps. This is where node.js usually wins

One use-case that ends up making multithreaded worse is when you need to run another scripting language in your thread. First, you usually need to malloc the entire runtime for that language, then you need to malloc the variables used by your script.

Since neither architectures are not without tradeoffs some intelligent people came up with different thoughts and so a few of the single-threaded architectures use a hybrid approach. For example, instead of launching and handling different threads from a single process, you can launch multiple applications — for example, 4 Nodejs servers on a quad-core machine. Then you use a load balancer to spread the workload amongst the processes called cluster mode in Node.

The official documentation explains it quite nice, and it says

A single instance of Node.js runs in a single thread. To take advantage of multi-core systems, the user will sometimes want to launch a cluster of Node.js processes to handle the load.

But we are jumping too far ahead, so comping back to Node concurrency

One thread to rule them all

Node as you might have guessed already took a slightly different approach to handle multiple concurrent requests at the same time when compared to other popular servers like Apache. Creating a different thread for each request gets expensive very fast. Also, threads would be doing nothing while awaiting results from other operations like the one explained above (i.e.: a database read). That’s why Node is using one thread instead. Such an approach has numerous advantages mainly being:

1. No overhead when spawning new threads.

2. Your code is much easier to reason about, as you don’t have to worry about what will happen if two threads access the same variable because that simply cannot happen.

But there are some drawbacks as well. Node isn’t the best choice for applications that mostly deal with CPU-intensive computing(but worker threads are a choice here). On the other hand, it excels at handling multiple I/O requests. So, let’s focus on this part for a bit but if you wanna learn how to handle computing-intensive tasks in Nodejs like sorting a huge list look into cluster mode and worker threads

I/O operations and Node

Let me first answer the question: What the hell do you mean by I/O operations cause I don’t remember….. well, giving anything to it from the command to run?

Well, these are operations that communicate with stuff from the outside of your application like HTTP requests, disk reads, and writes or operations on the database, just to name a few. I/O in Node comes in two “flavors”: blocking and non-blocking. If you wanna be a great Node developer you gotta be able to distinguish between the two.

“Blocking” in “blocking I/O operations” is quite self-descriptive. It means that the next operations are blocked and that they have to wait as long as the currently running operation is taking. Let’s take a look at an example:

const fs = require('fs');
const data1 = fs.readFileSync('/file_1.md');
const data2 = fs.readFileSync('/file_2.md');
console.log(data);
moreWork(); // will run after console.log

The reason you are told to write multiple async/await functions instead of one big async/await is because what you are writing is blocking code(Remember this! It is a very important tip)

“Non-Blocking” in “non-blocking I/O operations” is also quite self-descriptive. It means that the next operations are not blocked and that they do not have to wait while the current operation is not running. Let’s take a look at an example:

const fs = require('fs');
fs.readFile('/file_1.md', (err, data) => {
  if (err) throw err;
  console.log(data);
});
moreWork(); // will run before console.log

Back to the original question of “If Node.js is single-threaded, how does it manages the concurrency?”

What you might be thinking is that most of the processing is handled in the node event loop. But Node actually farms off the I/O work to threads. I/O operations typically take orders of magnitude longer than CPU operations so why have the CPU wait for that? And thus Node achieves a much better CPU utilization

By way of analogy, think of NodeJS as a waiter taking the customer's orders while the I/O chefs prepare them in the kitchen. Other systems have multiple chefs, who take a customer's order, prepare the meal, clear the table and only then attend to the next customer.

That’s it. Now, you have truly reached the realm where you can call yourself a true NODEJS Developer without anyone calling “B***SH*T”.

Bonus:

I feel extra generous today so let me drop a few more popular Nodejs interview questions.

  1. What is the difference between let, const, and var ( Hint: Scope)
  2. What is the difference between forEach and map ( Hint: Modifies Original array)
  3. What is the difference between async/await, promises, and callbacks

I know it was an arduous journey from the start of this article to finishing it here(Trust me I know cause I wrote it), so pat yourself in the back, have some coffee, put a smile on that pretty face, and drop me a follow.

References & Resources:

How, in general, does Node.js handle 10,000 concurrent requests?
I understand that Node.js uses a single-thread and an event loop to process requests only processing one at a time (which is non-blocking). But still, how does that work, lets say 10,000 concurrent
How does node process concurrent requests?
I have been reading nodejs lately, trying to understand how it handles multiple concurrent requests, I know nodejs is a single threaded event loop based architecture, at a given point of time only ...
Can two http request come together? If it can, how nodeJS server handles it?
Yesterday I was giving some presentation on nodeJS.Some one asked me following question: As we know nodeJS is a single threaded server, several request iscoming to the server and it pushes all r...
A simple guide to JavaScript concurrency in Node.js | TSH.io
I bet that you are familiar with JavaScript concurrency in Node.js. Also, most probably you have already heard
Node.js multithreading: What are Worker threads, and why do they matter? - LogRocket Blog
Node.js v10.5 introduced the worker_threads module, and it has been stable since Node.js v12 LTS. But what are Worker threads, and why would you need them?
Don’t Block the Event Loop (or the Worker Pool) | Node.js
Node.js® is a JavaScript runtime built on Chrome’s V8 JavaScript engine.