62 Commits

Author	SHA1	Message	Date
Sebastian Raschka	9a5d4d8ac9	Try windows runners (#133) ... * try windows runners * update triggers * trigger with code file update * add new status badges	2024-04-28 07:39:23 -05:00
Sebastian Raschka	4adb96d7ee	Make code more consistent and add projection layer (#131) ... * Make code more consistent and add projection * remove redundant buffer	2024-04-26 17:13:08 -05:00
Sebastian Raschka	7740d556a0	Use dim=-1 for consistency (#122)	2024-04-18 05:56:23 -05:00
James Holcombe	05718c6b94	Use instance tokenizer (#116) ... * Use instance tokenizer * consistency updates --------- Co-authored-by: Sebastian Raschka <mail@sebastianraschka.com>	2024-04-10 21:16:19 -04:00
rasbt	6de0417321	cleanup	2024-04-04 07:58:41 -05:00
Sebastian Raschka	2de60d1bfb	Rename variable to context_length to make it easier on readers (#106) ... * rename to context length * fix spacing	2024-04-04 07:27:41 -05:00
rasbt	7d1eadd0be	update notes	2024-04-02 18:27:13 -05:00
Intelligence-Manifesto	96b1fde3f1	"Typographical error (#104)	2024-04-02 18:07:21 -05:00
rasbt	3ad442ee90	skip version cell	2024-03-28 08:23:33 -05:00
Sebastian Raschka	cf39abac04	Add and link bonus material (#84)	2024-03-23 07:27:43 -05:00
Sebastian Raschka	a2cd8436cb	Ch05 supplementary code (#81)	2024-03-19 09:26:26 -05:00
Sebastian Raschka	ca96abac8a	Set up basic test gh worklows (#79) ... * Set up basic test gh worklows * update file paths * env check * add env check * Update requirements.txt * simplify * upd	2024-03-18 11:58:37 -05:00
Sebastian Raschka	9d6da22ebb	Update pep8 (#78) ... * simplify requirements file * style * apply linter	2024-03-18 08:16:17 -05:00
rasbt	4fc6de7afa	add notes	2024-03-17 09:29:06 -05:00
rasbt	d60da19fd0	add more notes and embed figures externally to save space	2024-03-17 09:08:38 -05:00
Intelligence-Manifesto	d4b4e3d0f0	the above -> the following	2024-03-15 05:00:28 +08:00
rasbt	1870b4bacd	update stride param	2024-03-13 08:39:59 -05:00
rasbt	0d517e98b9	update	2024-03-13 08:37:54 -05:00
rasbt	f2c8eeb6b8	pretraining on project gutenberg	2024-03-13 08:34:39 -05:00
rasbt	569f6bc7f0	benchmark numbers	2024-03-13 07:12:10 -05:00
taihaozesong	f1fa9df15c	Fix mha wrapper implementations in ch03 bonus	2024-03-13 18:02:26 +08:00
rasbt	321f3d33f9	add cuda warmup	2024-03-10 10:31:55 -05:00
rasbt	244137e8a1	amend	2024-03-10 08:05:22 -05:00
rasbt	76205521d7	different dropout behavior on macos and linux	2024-03-10 07:58:10 -05:00
rasbt	73822b8bfa	move ex 3.3 solution outside main chapter	2024-03-10 07:18:24 -05:00
rasbt	da33ce8054	remove redundant unsqueeze in mask	2024-03-09 17:42:31 -06:00
rasbt	6ba97adaee	add PyTorch version	2024-03-09 17:42:30 -06:00
rasbt	5ca60321c4	add a100 numbers	2024-03-09 10:20:08 -06:00
rasbt	29ca41799a	use need_weights=False	2024-03-09 10:09:17 -06:00
rasbt	5643c88db9	add pytorch mha	2024-03-08 09:30:55 -06:00
rasbt	3beaea46ce	add lowres figs for better navigation	2024-03-08 07:18:06 -06:00
rasbt	404f48aa74	automatically run on gpu or cpu	2024-03-07 20:14:03 -06:00
rasbt	99a5e28def	rename q,k,v for consistency with chapter 3	2024-03-07 06:30:40 -06:00
Rayed Bin Wahed	496079c61e	Update mha-implementations.ipynb ... Fix variable spelling in comments to keep consistent with code	2024-03-06 23:03:57 +08:00
rasbt	b6fe1a37b3	also add simple wrapper	2024-03-06 08:38:53 -06:00
rasbt	571377a2d6	update title	2024-03-06 08:34:04 -06:00
rasbt	87fcfd9245	mha variants	2024-03-06 08:30:32 -06:00
rasbt	d4754f1bdd	change dim=1 to dim=-1	2024-03-04 18:54:43 -06:00
rasbt	b827bf4eea	remove redundant double-unsequeeze	2024-02-29 08:31:07 -06:00
rasbt	250e6306e2	use attn_scores from sec 3.4 instead of 3.3	2024-02-14 20:23:59 -06:00
Intelligence-Manifesto	6a09e7b03a	**step 2** ... step 2: According to the context, the formatting here should be **step 2**. Additionally, it seems that there is a lack of text description for step 1 in this section, as other sections are all labeled with steps 1, 2, 3 in order, clearly indicating the steps.	2024-02-12 18:32:28 +08:00
Intelligence-Manifesto	1278615c25	12 -> 21 ... 12 -> 21	2024-02-11 14:17:55 +08:00
rasbt	3a5fc79b38	add and update readme files	2024-02-05 06:51:58 -06:00
rasbt	8860e16e05	<|endoftext|> token in dataset v1	2024-01-21 12:03:04 -06:00
rasbt	92896d817c	add toggle for qkv_bias	2024-01-17 07:50:57 -06:00
rasbt	dfe2c3b46f	use blocksize in positional embedding	2024-01-15 08:15:33 -06:00
rasbt	9e85f13ba9	readability improvements	2024-01-15 07:36:19 -06:00
rasbt	a7b4880179	small readability updates	2024-01-14 11:58:42 -06:00
rasbt	c79499572f	update chapter title	2024-01-13 14:51:39 -06:00
rasbt	c400f77f26	update exercise solutions	2024-01-13 14:49:02 -06:00