Adding IPFS File Storage to a Web App (Node.js + MongoDB + Fleek)

Don't use expensive CDNs when you can host your images and files on IPFS in just a few clicks. Fleek makes that possible. Here's how!

Adding IPFS File Storage to a Web App (Node.js + MongoDB + Fleek)

So I wanted to add images to the profiles on Coopefy but didn't want to do it through the usual methods (S3, CDNs, etc) so I ended up using MongoDB at first.

This approach wasn't the best, given databases aren't designed to work with files, especially images.

Seeing MongoDB wouldn't be a great idea for scaling later on, I was looking for a better way to upload these images, but instead of using the typical methods, I wanted a more decentralized approach (IPFS).

At first, it felt like a difficult thing to do as IPFS seems way too good to be true. Then I found out it was actually possible and cool...

FREE STORAGE FOR ALL THE FILES I WANT?
WITH DECENT PERFORMANCE?
ARE YOU KIDDING ME?!

I was happy (and deeply surprised) with the fact it does not only work but it's a lot faster than I thought too.

Before starting be aware that I'm using:

NodeJS + Express = for back-end development (mainly connecting with database)

MongoDB = database (some of the commands such as findOneandUpdate are unique to Mongo)

Fleek = this is my IPFS host, where all my files (pictures in this case) are saved.

With that in mind, here's how you can do it:

1. Create a Fleek.co Account

There are many IPFS hosting sites, but few are as popular as Fleek. You can also try Infura and Pinata if you want, both are known for being reliable and useful (they may or not be free, so check that out before proceeding).

Anyway, Fleek has a feature you can use to store general files. The feature comes with a pre-selected bucket typically named after you + team bucket (e.g, angel-team-bucket in my case) where every file drops into as you upload it.

After signing up either with your name+email or Github account, you'll be redirected to a dashboard. Here, you'll find a menu on the left, like this:

Sure enough, just click on Storage and you'll be taken into the feature I'm talking about.

In this page, you can create more folders as necessary so you can store your files in an organized way.

NOTE: I couldn't find any comment about storage limit from Fleek, so you're likely getting unlimited IPFS storage. Given how decently quickly and convenient it is, this is a real game-changer.

2. Add Fleek to Your Javascript Application

Once you're done setting up a Fleek's account, the process is very straightforward as the app comes with a pretty decent guide to follow.

TL;DR:

You install the package this way:

npm install @fleekhq/fleek-storage-js

The package installs quickly, so you can proceed to import the package into your app in either of these ways. You can wait for the installation to finish if you prefer.

Once installed, you can import it to your React front-end using:

import fleekStorage from '@fleekhq/fleek-storage-js'

For NodeJS, it would be like this (this is how I use it on my back-end):

const fleekStorage = require('@fleekhq/fleek-storage-js')```

3. Use the upload function from Fleek's

Once again, Fleek's documentation shows clearly how to upload a file to your IPFS bucket.

Here's how it goes for a back-end connected to MongoDB (the imageUpload function is an utility I'm using to process images so they're converted to buffer before reaching the IPFS storage, so you can ignore that):

 router.post('/your-route', imageUpload.single('profileimage'),

  async (req, res) => {

    try {

      const uploadedFile = await fleekStorage.upload({
      apiKey: process.env.API_KEY,
      apiSecret: process.env.API_SECRET,
      key: whatever-you-want-to-use-as-key,
      data: req.file.buffer,
      bucket:'yourname-team-bucket/optional-folder',
      httpUploadProgressCallback: (event) => {
        console.log(Math.round(event.loaded/event.total*100)+ '% done');
      }
    });

As you can see on the uploadFile function above, Fleek offers several inputs. The documentation explains them clearly here:

The bucket refers to the name of the place they give you to store files (e.g, angelgomez-team-bucket in my case). You can also create multiple sub-folders on that bucket via the Fleek browser app, and add them to the upload function using the same bucket URL (e.g, angelgomez-team-bucket/new-bucket).

To change the bucket folder, you would look have to write something like this on the uploadFile function:

bucket: angelgomez-team-bucket/new-bucket

NOTE: If you aren't sending images, it's recommended that you install the fs package first and then, before invoking the uploadFile function, you read the file that's being sent.

Here's an example:

fs.readFile(filePath, async (error, fileData) => {
  const uploadFile = .....
  )}```

This way IPFS handles it as a file and it is easier for you to fetch it later without issues.

4. Get the Responses and Send to Database

The uploadFilefunction above returns several things:

1. You will find the hash of the file which is a random string (access via uploadedFile.hash)

2. The publicUrl that gives direct access to the file you're uploading (access it via uploadedFile.publicUrl)

3. And the key to the bucket which will end up being the name of the file (access via uploadFile.key)

My upload-image function looks like this:

  let function = await MongoSchema.findOneAndUpdate(
        { user: req.user.id },
        { profileimage: uploadedFile.publicUrl })

      res.send(profile.profileimage);```

As you can see in the function above, I'm using the uploadFile.publicUrl to send the image address (IPFS file link) to the MongoDB database (using res.send).

This way, I'm not saving any files directly on the database (which is obviously not ideal) but instead  saving the URL of the file from IPFS so I can retrieve it later.

5. Fetching the Images on the Front-End

Because the images are saved as buffer on IPFS (you can download them and add a .jpeg, .jpg, or .png extension to open them) they're also easy to render in the front end.

After fetching the image file from the database (saved as Binary), I use this function to transform the binary file into renderable buffers:

const fileContents = new Buffer(profileimage, 'base64');
setImagePrev(fileContents);

In this function above, the imagePrev works as the state where I'm saving the file after reading it as a buffer (Binary data). This imagePrev is what renders the image on the front-end using a <imgsrc={imagePrev}/>.

Questions You May Have (QYMH)

How long will setting up an IPFS bucket with Fleek take me?

It took me no more than 20 minutes to set up the bucket and about 1 hour to figure out how to add it into my app. With this guide, you may get everything done within 15 minutes or less if you're quick.

Is IPFS performance similar to that of AWS?

Probably not. If you've worked with CDNs or AWS S3 before, you'll realize IPFS is a lot slower as you're fetching files (in Coopefy's case, images).

But it's still surprisingly fast. In fact, you may not care as long as speed isn't a major feature of your application.

Is Fleek's API compatible with SDKs?

Yes. In fact, you can see it as a main feature at Fleek's Storage landing page.

Now that's pretty much everything. This guide is probably not necessary if you're familiar with typical APIs but I wanted to make it more accessible for those who aren't.

Anyway, thank you for reading :)