i agree. the llm is like the brocas area. it has generative grammar and semantic categories but there should be a separate model that checks relevant corpuses for agreement. the only issue would be the large energy and time costs at runtime. hence why they try to do both in the llm, i think.

Then wouldn't that make the whole tool redundant?

@@Luixxxd1 Pretty much ...Its more effecient to run specialzed agent to check math when needed than blob everything together at runtime.

We are performing research on using LLMs and a part of that research is safety related i.e. making sure that our models are truthful, useful, and robust to adversarial attacks. However, we are not focused on security specifically.

@@nPlan good, my thesis is on LLM security and I was looking for any masters/ PhD researcher for a possible collaboration of sorts.

that is true, do you have any resources you would suggest to learn more about the topic? We choose papers for paper club that we find interesting but are not necessarily our research background, so we cannot always give an in-depth presentation of the topic.

Empirical distribution are the training data, it will be a mixture of point masses (look up 'Dirac delta') at the locations of the samples in the sample space. Then match forward and reverse markov chains that go from a p_theta(x, t=0) to a normal distribution at t=T which gives you a nice denoising score-matching objective that can be used to train energy based models (train p_theta(x, t)) or score based models (train grad_{x}(p_theta)(x, t)). This training is done by noising samples from the empirical distribution and predicting the amount of noise added. Inductive bias or regularisation gives inaccurate score after training resulting in that you don't recover empirical distribution after training but something more desirable to practitioners that can generalise and achieve good results on metrics they are interested in, such as FID score.

@@benboys_thank you very much!

I was wondering because I saw an explanation that said we need Langevin Dynamics for sampling from the model, such that those samples can then be used in a MCMC estimator for the true likelihood of the model.

I would think generally not. I mean that is your most general data science problem, how do you predict stuff that is wildly different from the past data that you have seen. Now, that doesn't mean that it isn't somewhat tracking, since the latest data does go in and most forecasts will be somewhat guided by the latest "is" state. The fact that you typically predict over many entities, you would also hope that some generalisation would naturally occur as some similarities have been already observed in other entities. Now these models are not truly causal, at least not out of the box. And they can't, because causality cannot be inferred from data (caveats caveats caveats). So it does fall back to the modeller to provide sensible covariates. As long as causal pathways don't break down (and sometimes that can happen, at least temporarily) the model generalises. If one only throws features at the model and hopes it finds all it needs by itself, one might be in for a bad surprise.

github.com/kyegomez/RT-2

Yes, you're right, same thing up to a sign change and people usually refer to convex optimization or log concave sampling (of a probability density)

Based on my experience with reinforcement learning problems, the reward function is often a heuristic or a tuned hyperparameter. I think the authors chose the reward function in section 4.1 because by changing its weighting values they are able to change the difficulty of the learning problem. So while the reward function does not have a theoretical justification, it makes sense in terms of experimental design to illustrate an idea in the paper which is that GFlowNets are able to solve harder RL problems. I think there are two properties which mainly characterize reward functions. All reward functions output a real number which is used as the training signal for RL. One property of reward functions are dense versus sparse. A dense reward function is one provides different rewards for each state, action pair. A sparse reward function is usually one that provides a reward of 0 except when a goal or keypoints towards a goal are reached and then some positive reward is received by the agent (usually 1). A dense reward function makes it easier for an agent to learn to accomplish a task, but usually make it harder for the agent to overcome bottlenecks in the task. A sparse reward function provides the agent with the most accurate training signal for learning to accomplish a task, but may provide very few rewards and consequently very little training signal to learn from. The other property of reward functions is stochastic versus deterministic where a stochastic reward function determines its reward for a state action pair by sampling from some distribution it outputs as an intermediary step in the function. The researcher / engineer using reinforcement learning has to choose / design the reward function, if they choose to use a dense reward function like that in section 4.1. I hope this information was helpful!

Just search for nPlan paper club. We do weekly discussions open to the public and in-person in London once a month!

Thanks for your explanation about what assumptions are required to make a process reversible! I think it is also useful to understand the functional form of the transition density through the lens of Bayes rule on the reverse transition density and using the Taylor expansion (e.g., see the paper "Diffusion Schrödinger Bridge with Applications to Score-Based Generative Modeling" eq. (3) and eq. (4)), since we always compute in discrete time. This also uses the small step size limit and conjugacy property of the forward Gaussian and the Gaussian term resulting from the truncated Taylor expansion, but has the advantage of being interpretable algorithmically, as well as theoretically. A couple other points of discussion I picked up from this video: Arvid's question at 21:35, which I paraphrase 'why the reverse process seems to denoise if it is a stochastic transition', Nayef and Arvid correctly come to the answer that it is because the mean function of the reverse transition density counteracts or 'wins out' over the diffusion, by forcing the particles to areas of high probability. 27:50, 29:40 The reverse step leads to a distribution that is not isotropic Gaussian because the mean function is parameterized by a neural network, which is nonlinear in the state, leading to a non-Gaussian distribution from the first step onwards. 28:20, 38:25 In practice, the forward chain does not quite reach an isotropic Gaussian. An approximation is made by sampling from the isotropic Gaussian in the first step of the reverse chain. I think that the initial sample of the reverse chain is not independent from the final denoised sample as Peter hints to at 28:49. I think this is complicated to show and depends on the number of steps, and the parameterization of forward chain. Usefully, running parallel chains independently will lead to independent samples from the reverse chain.

Sorry for the delay, you can sign up to our up and coming Paper Clubs by following this link www.meetup.com/home/?suggested=true&source=EVENTS

Hello, Were you able to find the slides from this video?

@@divugoel no I haven't. Still hoping someone with them will see this.

Thanks for your interest! nPlan's paper club is open for all to attend in person in London or online! Just search for nPlan paper club :)