What’s a Webhook?

A webhook (also called a web callback or HTTP push API) is a way for an app to provide other applications with real-time information. A webhook delivers data to other applications as it happens, meaning you get data immediately. Unlike typical APIs where you would need to poll for data very frequently in order to get it real-time. This makes webhooks much more efficient for both provider and consumer. The only drawback to webhooks is the difficulty of initially setting them up.

Screen Shot 2018-02-02 at 6.48.22 PM

Webhooks are sometimes referred to as “Reverse APIs,” as they give you what amounts to an API spec, and you must design an API for the webhook to use. The webhook will make an HTTP request to your app (typically a POST), and you will then be charged with interpreting it.

Consuming a Webhook

The first step in consuming a webhook is giving the webhook provider a URL to deliver requests to. This is most often done through a backend panel or an API. This means that you also need to set up a URL in your app that’s accessible from the public web.

The majority of webhooks will POST data to you in one of two ways: as JSON (typically) or XML (blech) to be interpreted, or as a form data (application/x-www-form-urlencoded or multipart/form-data). Your provider will tell you how they deliver it (or even give you a choice in the matter). Both of these are fairly easy to interpret, and most web frameworks will do the work for you. If they don’t, you may need to call on a function or two.

Important Gotchas

There are a couple things to keep in mind when creating webhook consumers:

  • Webhooks deliver data to your application and may stop paying attention after making a request. This means if your application has an error your data may be lost. Many webhooks will pay attention to responses and re-send requests if your application errors out. If your application processed the request and still sent an error, there may be duplicate data in your app. Understand how your webhook provider deals with responses so you can prepare for the possibility of application errors. Additionally, you may want to check out our tool Reflector.io for help dealing with webhook errors and queuing.
  • Webhooks can make a lot of requests. If your provider has a lot of events to tell you about, they may end up DDoSing your app. Make sure your application can handle the expected scale of your webhook. We made another tool, Loader.io, to help with that.

Get Your Feet Wet

The best way to truly understand a webhook is to try one. Luckily, lots of services use webhooks so you can easily play with them to your heart’s content. Check out some of the webhooks below:

Advertisements

SSH Passwordless Login Using SSH Keygen

First log in on A as user a and generate a pair of authentication keys. Do not enter a passphrase:

a@A:~> ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/a/.ssh/id_rsa): 
Created directory '/home/a/.ssh'.
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/a/.ssh/id_rsa.
Your public key has been saved in /home/a/.ssh/id_rsa.pub.
The key fingerprint is:
3e:4f:05:79:3a:9f:96:7c:3b:ad:e9:58:37:bc:37:e4 a@A

Now use ssh to create a directory ~/.ssh as user b on B. (The directory may already exist, which is fine):

a@A:~> ssh b@B mkdir -p .ssh
b@B's password: 

Finally append a’s new public key to b@B:.ssh/authorized_keys and enter b’s password one last time:

a@A:~> cat .ssh/id_rsa.pub | ssh b@B 'cat >> .ssh/authorized_keys'
b@B's password: 

From now on you can log into B as b from A as a without password:

a@A:~> ssh b@B

A note from one of our readers: Depending on your version of SSH you might also have to do the following changes:

  • Put the public key in .ssh/authorized_keys2
  • Change the permissions of .ssh to 700
  • Change the permissions of .ssh/authorized_keys2 to 640

Python: “json.dumps()” to pretty-print Python dicts

# The standard string repr for dicts is hard to read:
>>> my_mapping = {'a': 23, 'b': 42, 'c': 0xc0ffee}
>>> my_mapping
{'b': 42, 'c': 12648430. 'a': 23}  # 😞

# The "json" module can do a much better job:
>>> import json
>>> print(json.dumps(my_mapping, indent=4, sort_keys=True))
{
    "a": 23,
    "b": 42,
    "c": 12648430
}

Especially when you print dicts in logs for rest apis, where in you need to reuse them, it is always good practice to use json.dumps and print them as json instead of dict.

What is Memcached ?

As name suggested Memcached is cache which could be though of as big key value pair bucket residing on RAM which can deliver frequently used data instantly by avoiding datasource access.

Memcached is a general-purpose distributed memory caching system. It is often used to speed up dynamic database-driven websites by caching data and objects in RAM to reduce the number of times an external data source (such as a database or API) must be read.

Memcached’s APIs provide a very large hash table distributed across multiple machines. When the table is full, subsequent inserts cause older data to be purged in least recently used (LRU) order.[3][4] Applications using Memcached typically layer requests and additions into RAM before falling back on a slower backing store, such as a database.

Converting database or object creation queries to use Memcached is simple. Typically, when using straight database queries, example code would be as follows:

 function get_foo(int userid) {
    data = db_select("SELECT * FROM users WHERE userid = ?", userid);
    return data;
 }

After conversion to Memcached, the same call might look like the following

 function get_foo(int userid) {
    /* first try the cache */
    data = memcached_fetch("userrow:" + userid);
    if (!data) {
       /* not found : request database */
       data = db_select("SELECT * FROM users WHERE userid = ?", userid);
       /* then store in cache until next get */
       memcached_add("userrow:" + userid, data);
    }
    return data;
 }

The client would first check whether a Memcached value with the unique key “userrow:userid” exists, where userid is some number. If the result does not exist, it would select from the database as usual, and set the unique key using the Memcached API add function call.

However, if only this API call were modified, the server would end up fetching incorrect data following any database update actions: the Memcached “view” of the data would become out of date. Therefore, in addition to creating an “add” call, an update call would also be needed using the Memcached set function.

 function update_foo(int userid, string dbUpdateString) {
   /* first update database */
    result = db_execute(dbUpdateString);
    if (result) {
       /* database update successful : fetch data to be stored in cache */
       data = db_select("SELECT * FROM users WHERE userid = ?", userid);
       /* the previous line could also look like data = createDataFromDBString(dbUpdateString); */
       /* then store in cache until next get */
       memcached_set("userrow:" + userid, data);
    }
 }

This call would update the currently cached data to match the new data in the database, assuming the database query succeeds. An alternative approach would be to invalidate the cache with the Memcached delete function, so that subsequent fetches result in a cache miss. Similar action would need to be taken when database records were deleted, to maintain either a correct or incomplete cache.

What is ISO date format

As world is becoming a big family with globalization, we need the standard to followed to avoid confusion especially related to date and time. If every country or person would write dates in different format, it would be confusing to figure out exact date while doing communication across countries, hence ISO has come up with ISO date format to be followed by every one. 

International Standard ISO 8601 specifies numeric representations of date and time. This standard notation helps to avoid confusion in international communication caused by the many different national notations and increases the portability of computer user interfaces.

YYYY-MM-DDThh:mm:ss.sTZD

where:

     YYYY = four-digit year

     MM   = two-digit month (01=January, etc.)

     DD   = two-digit day of month (01 through 31)

     hh   = two digits of hour (00 through 23) (am/pm NOT allowed)

     mm   = two digits of minute (00 through 59)

     ss   = two digits of second (00 through 59)

     s    = one or more digits representing a decimal fraction of a second

     TZD  = time zone designator (Z or +hh:mm or -hh:mm)

example: 

 1997-07-16T19:20:30.45Z (Z at end indicates time is taken with Zero correction i.e. UTC time)

 1997-07-16T19:20:30.45+01:00 (+1:00 at end indicates time is taken at time zone which is 1:00 ahead of UTC time)

Advantages of the ISO 8601 standard date notation compared to other commonly used variants:

  • easily readable and writeable by software (no ‘JAN’, ‘FEB’, … table necessary)
  • easily comparable and sortable with a trivial string comparison
  • language independent
  • can not be confused with other popular date notations
  • consistency with the common 24h time notation system, where the larger units (hours) are also written in front of the smaller ones (minutes and seconds)
  • strings containing a date followed by a time are also easily comparable and sortable (e.g. write “1995-02-04 22:45:00”)
  • the notation is short and has constant length, which makes both keyboard data entry and table layout easier
  • identical to the Chinese date notation, so the largest cultural group (>25%) on this planet is already familiar with it 🙂
  • date notations with the order “year, month, day” are in addition already widely used e.g. in Japan, Korea, Hungary, Sweden, Finland, Denmark, and a few other countries and people in the U.S. are already used to at least the “month, day” order
  • a 4-digit year representation avoids overflow problems after 2099-12-31

How to write and execute mongoDB scripts

There are times when we need to save the steps or commands of mongo shell and need to get it executed in the same order (for reusability and automation and avoid errors). The best solution is to save it in a file with any extension preferred is js, however I call it as mjs which means mongo javascript. With below command you can run it in bash

aseem278$ mongo < /Users/asee2278scripts/mongoScript.mjs

TIP : To be effective while writing query, you write commands using IDE to use auto complete, formatting, Json format verification and then redirect input to mongo 

Sample mongoScript.mjs

use school

db.scores.drop()

var types = [‘exam’, ‘homework’, ‘quiz’]

for (student_id = 0; student_id < 100; student_id++) {

    for (type=0; type < 3; type++) {

   var r = {‘student_id’:student_id, ‘type’:types[type], ‘score’:Math.random() * 100};

   db.scores.insert(r);

    }

}