Web Security Essentials Live Workshop with Mike Sherov Part 1

node

Web Security Essentials Live Workshop 2019-12-03 Part 1

Web Security Essentials Live Workshop with Mike Sherov - Part 2

Web Security Essentials Live Workshop 2019-12-03 Part 2

Web Security Essentials Live Workshop with Mike Sherov - Part 3

Web Security Essentials Live Workshop 2019-12-03 Part 3

egghead

Live Workshop Recording

vitest is a versatile testing tool that provides various options to run tests. Here's a breakdown of how you can use it:

**Running Tests in Watch Mode**

By default, vitest runs in watch mode. This means that the tests will re-run automatically whenever there are changes in your functions.

For example, if you have a script defined in your package.json as:
```json
{
  "scripts": {
    "test": "vitest"
  }
}
```

You can run it with the following command:

```bash
npm run test
```

If your function fails and you update it, vitest will run again, giving you immediate feedback.

**Running Tests Once**

If you wish to run the tests just once without entering the watch mode, you can use the `vitest run` command.

You can add this to your package.json scripts as:

```json
{
  "scripts": {
    "test": "vitest",
    "test:once": "vitest run"
  }
}
```

And then run it with:
```bash
npm run test:once
```

**Accessing the vitest UI**

vitest comes with a built-in UI which can be accessed using the `--UI` flag. 

In your package.json, you can have a script like:
```json
{
  "scripts": {
    "test": "vitest",
    "test:ui": "vitest --ui"
  }
}
```

To launch the dashboard along with the tests in watch mode, use:
```bash
npm run test:ui
```

![vitest ui](https://res.cloudinary.com/dg3gyk0gu/image/upload/v1698440916/tips/kevin-cunningham/vitest-ui.png)

This dashboard provides insights into:

each test and its report:

![single test](https://res.cloudinary.com/dg3gyk0gu/image/upload/v1698440916/tips/kevin-cunningham/vitest-single-test.png)

the module graph:

![module graph](https://res.cloudinary.com/dg3gyk0gu/image/upload/v1698440916/tips/kevin-cunningham/vitest-module-graph.png)

the code where the test originates from:

![code tested](https://res.cloudinary.com/dg3gyk0gu/image/upload/v1698440917/tips/kevin-cunningham/vitest-code-test.png)

**Passing Flags to Package Scripts**

When you want to pass additional flags to the script with `npm run`, you might encounter an issue. For instance, running:
```bash
npm run test --ui
```

Won't work as expected. This is because the `--UI` flag is being passed to npm and not to the vitest command.

To correctly pass the flag, use an additional double dash `--` before your flags, like so:
```bash
npm run test -- --ui
```

This tells npm that the subsequent flags should be passed to the script and not npm itself.

In conclusion, vitest offers a rich set of features to help you run and monitor your tests effectively. Whether you prefer the watch mode for continuous feedback or need the UI dashboard for detailed insights, vitest has you covered. Remember to use the double dash `--` when passing flags to your npm scripts to ensure they work as intended.

https://app.egghead.io/api/v1/lessons/node-js-web-security-essentials-live-workshop-2019-12-03-part-1/subtitles

Enhance Your Testing Workflow with vitest

[00:00] When I have a package script like this I can run it with npm run test. vitest runs in watch mode automatically so that if I change my function to fail it will run if I update it will run again.

[00:18] If instead I only wanted this to run one time I can run vitest run. In my package script I could add a another script tag vitest run like that and then npm run test once. It'll run once and then quit.

[00:37] vitest also comes with a UI which we can access with a flag dash dash UI. So I'll add another script vitest dash dash UI and I'll run npm run test UI which as well as running my test in watch

[00:53] mode will launch this dashboard where I can look at each test its report the module graph and the code where the test goes from. Creating these multiple test functions just to pass these parameters can be a bit frustrating especially for parameters that you're only using once or twice.

[01:13] If I do npm run test run then that will do the same as test once so we can get rid of that one. If I do npm run test dash dash UI that does not launch my UI and that's because

[01:30] this dash dash UI flag is being passed to npm and not to the script that npm run test is pointing towards. If I do want to pass additional flags to this script in other words reproduce this vitest

[01:45] dash dash UI then before my flags I do a separate dash dash. This is telling npm that anything that comes after this point shouldn't be passed to npm but should instead be appended to the end of the

[01:59] script tag. So this npm run test dash dash dash dash UI is the same as vitest dash dash UI so it's the same as this. So if I run this as well as running my tests I also get my dashboard. So you can pass arguments to package scripts in line but to be able to pass

[02:16] flags to package scripts you have to use an additional double dash.

https://app.egghead.io/api/v1/lessons/node-js-web-security-essentials-live-workshop-2019-12-03-part-1/transcript

I recently stumbled upon a fantastic tool for authentication: **Clerk**. Here's why I think it's a game-changer:

### 1. **Easy Installation and Setup**
By simply following their outstanding documentation, you can install the necessary packages and set up Clerk in no time. If you're using AppRouter for Next.js, all you need to do is wrap your entire layout with `ClerkProvider`, and voila – instant authentication!

```jsx
<ClerkProvider>
  {/* Your App's Components */}
</ClerkProvider>
```

![clerk auth](https://res.cloudinary.com/dg3gyk0gu/image/upload/v1698429385/tips/alejandro-roman/clerk_auth.png)

### 2. **Out-of-the-box Login and Sign-up Screens**
Gone are the days of building a login and sign-up screen from scratch. With Clerk, you get these, along with social media sign-up options, right out of the box. Just use their provided `sign-up` and `sign-in` components, set some environment variables, and you're good to go.

### 3. **Robust Route Authentication**
Through middleware, you can utilize the `authMiddleware` to specify which routes are public and which ones to ignore. This means that any route that isn't public or ignored will require authentication – a feature that ensures enhanced security.

```jsx
// Using authMiddleware to specify routes

export default authMiddleware({
  publicRoutes: ["/api/documents/insights/1"]
})
```

### 4. **Access to User Details with getAuth**
By importing `getAuth` from Clerk's package, you can pass in the request object and instantly access the entire user's details. This functionality enables you to swiftly implement features like:

```jsx
const {userId} = getAuth(req);

if (!userId) {
  return new Response("Unauthorized", {status: 401});
}
```

### 5. **Exceptional Documentation**
I can't emphasize enough how comprehensive and user-friendly Clerk's documentation is. From installation guides to detailed explanations of features and components – everything is meticulously laid out. Plus, the option to copy-paste components means you can achieve a lot with minimal effort.

### 6. **User-Friendly Components**
Clerk also offers components like the `UserButton`, a handy UI element that lets users manage their accounts or sign out. The best part? You don't even need to build the sign-out functionality – it's all taken care of!

In conclusion, Clerk has transformed the way I approach authentication in my projects. Its ease of use, combined with its powerful features, makes it an invaluable tool for both seasoned developers and beginners. I've already incorporated it into several of my side projects over the past month, and I'm eagerly looking forward to using it in future endeavors.

Simplify Your Web App's Authentication Process with Clerk

[00:00] So, this is another tool that I'm really, really excited about, and it's for authentication. It's called Clerc. You install the packages. Their documentation is amazing, by the way. In your layout, if you're using AppRouter for Next.js, you just simply wrap the entire layout with ClercProvider, and you automatically get your authentication, right?

[00:19] You get, like, things like this out of the box, which is insane. The fact that you don't have to build a login screen anymore, a sign-up screen. You get, like, social media, sign-up, et cetera. And all I really did was, I went through the documentation, they even give you this component called sign-up, one for sign-in, and you add some environment variables, and all of a sudden,

[00:39] you have all that, right? Something else that's really cool is that, through middleware, you can instantiate what they call off-middleware, and you can declare what routes are public and what routes to ignore. So, basically, everything that is not public route or ignored will basically get locked

[00:58] out in authentication, right? Which is awesome. Another really cool thing is that you have this thing called getAuth, which you import from their package as well. And through getAuth, you give it the request object, and now you have access to the entire user. So, you can do really cool things, right?

[01:15] And this is just a very basic example, which is, like, okay, if there's no user ID, just respond that it's unauthorized and status 401. And that's insane that you could do so much in so little time. In the past, it would take me, like I said, days to build this, and now it literally took me, like, 30 seconds to install all this.

[01:33] Another really cool thing is that their documentation is on point. Like when I mean on point, like, it's awesome. Just go through it. You know, they give you how to install the package. You have to set this environment variables in your .env, or if you're using Doppler or

[01:50] like some environment variable provider, then you mount clerk provider by wrapping around your layout, or if you're using still pages, you know, you can wrap it around your app. And then it just goes on and explains all these things, right? It even gives you the components, like if you want to copy paste this, you did.

[02:10] That's what I did. And because I copy pasted this, I got this out of the box. This is insane. And you know, again, you can even, like, set up, like, okay, what's going to happen after signing through environment variables. Like, that's crazy. They have this little cool thing, too, that I really like. It's called, like, the user button.

[02:27] So it's like that little pill that has, like, manage account on top and, like, sign out and stuff. So you don't even have to build sign out functionality. Like, I'm so excited about this tool. I really loved it. I've used it for all my little, like, indie hacking side projects that I've built over the past month. And to be honest, like, I'm super stoked to continue using it.

https://app.egghead.io/api/v1/lessons/node-js-web-security-essentials-live-workshop-2019-12-03-part-2/transcript

Lets dive into a high-level explainer on how you can use TypeScript to chat with a document. For this, we'll be employing two main tools: **OpenAI** and **Pinecone**.

We'll have 4 main steps that I'm going to walk you through:


1. **Document Setup**: I've got a social media study in PDF format, about 75 pages, stored in S3.
   
2. **Data Extraction**: We download this document and use a library called `PDF parse` to extract its data. This data is then split by paragraph or a set character limit. We also have logic to remove any chunk that's 10 words or less because, let's be real, there's not likely much meaningful information in such short snippets.

3. **Creating Embeddings**: We then iterate over these chunks to create embeddings. I'll deep dive into what embeddings are in a more detailed video soon. These embeddings are prepared and upserted as vectors into Pinecone.

4. **Interacting with the Document**: When a user poses a question, we craft a new embedding from that prompt. This is used to create a query request which is then sent to the Pinecone index. Pinecone returns the top five results. We construct a context prompt from these, feed it to OpenAI, and obtain a response to present to the user.

![process image](https://res.cloudinary.com/dg3gyk0gu/image/upload/v1698427862/tips/alejandro-roman/chat_with_doc_process.png)

I've set up two endpoints in Postman that will interact with this document AI:
- **Summary**: This endpoint processes the document, splits the text into chunks, and upserts the data into Pinecone.
- **Chat**: Using this, we can ask questions against our processed document. It grabs a user's query, makes an embedding, and fetches the top results from Pinecone to generate a relevant answer.

https://app.egghead.io/api/v1/lessons/node-js-web-security-essentials-live-workshop-2019-12-03-part-3/subtitles

Chat with a Document Using TypeScript, OpenAI, and Pinecone

[00:00] Hey everyone, super excited to finally getting around to do this. I've been wanting to do some learning in public in video format. In the past I've created YouTube videos and tutorials in places such as Egghead and I'll probably do a longer format video for this but anyways just gonna dive right into it. This is a

[00:19] very high-level explainer of how you can use TypeScript to chat with a document. We're gonna use two tools to do that. We're gonna use OpenAI and we're gonna use Pinecone. I'll skip basically talking about or going in-depth in this

[00:35] video. I'm gonna probably create another video for YouTube which is a longer format and I'll dive deep into the code for that. But for this one we'll keep it short, high-level. We're gonna do three things. We're going to one is go through this chart of everything I have right now. I'm gonna explain what the two

[00:54] Postman endpoints that I've built and then kind of just dig a little bit through the code but very high-level. Right so I have this thing that I've kind of set up here and the first thing we're gonna do is by the way you can get the file however you want. I just already had a files in S3. I have some social

[01:13] media study that I found online PDF format about 75 pages long. I've put that in S3. Then what we're gonna do is we're gonna download that and we're gonna use a library called PDF parse to extract the data. Once we've extracted the data from this document we're gonna split it either by paragraph or character limit

[01:32] and I have some logic there to like remove things that are 10 words or less because what let's be honest there's probably nothing meaningful there. Also then we're gonna iterate over the chunks that we've created in this step and we're gonna create embeddings. Again I'll explain later in a longer format video what embeddings are if anyone wants to know. But then we'll prepare those

[01:51] embeddings and upsert them in as vectors into Pinecone. Then once the user comes around and ask the prompt or a question we're gonna create a new embedding from that prompt and we're gonna use that to create a query request and send it to the Pinecone index. Once it's in the Pinecone index, Pinecone is going to

[02:10] return us the five top results. We're gonna create an array of those top results. We're gonna join them and construct like a context prompt, feed it to OpenAI and generate a response for the user. Again it's basically chatting against the documents using the documents as a database. So quick

[02:27] explainer here, I've created two endpoints that I have in Postman. One is called summary and the other one's called chat. I've kind of built this question that we're gonna use. So the first question or the first step, the first endpoint here is gonna create the chunks and then create the

[02:44] embeddings and insert them into Pinecone. So we're gonna hit send and basically just quickly through the code what it's doing here is it's grabbing the file, the socialmedia.pdf from my S3, summarizing it. It's not really summarizing per se but it's creating chunks, splitting the text and

[03:03] upserting that into Pinecone. Then, please ignore this TS ignore, I'm just doing this very raw, we're gonna have a chat endpoint which is what's gonna be able to ask questions against this. And what this endpoint is gonna do is it's gonna grab that question that the user made, it's gonna create an embedding, then

[03:21] it's gonna basically write against everything that's in Pinecone to try to get a result, get the five top results and return them back, which we're gonna construct a prompt from that. We're going to basically give it a very high-level instruction prompt like answer the question sincerely, if you don't know

[03:38] the response just reply I don't know. We're gonna provide the chosen text that came back, we're gonna give that as context, then we're going to like add the prompt to the question and then give the answer. We're gonna then grab this constructive prompt that we have here and we're gonna feed it to OpenAI and

[03:56] then give the answer back to the user. So let's go back to Postman. The process should have been done, it's like super quick. Now we're gonna hit send here and we're gonna ask it a question. What is the impact of social media usage for high school students? So it can take like a couple seconds here, so just bear

[04:15] with me. Awesome! So the answer that it was or it's basically it went through all of Pinecone and said okay the impact of social media usage on high school students varies, the student found that social media is frequently used by high school students for both educational entertainment purposes etc etc and we

[04:34] can ask it pretty much any other question we want and if it doesn't know it's just gonna say I don't know. Alright thank you for tuning in, let me know what you think in the comments and if you do want me to provide like the github repo and stay tuned for a longer format video on YouTube. Thank you all!

https://app.egghead.io/api/v1/lessons/node-js-web-security-essentials-live-workshop-2019-12-03-part-3/transcript

Course Content