GSoC Week 8

A busy week

This was quite a busy week since I had to make sure the build is perfect as well as give ample time to my research since I need to meet some deadlines. I have been working to make sure the build is a success. In the later parts of the week I have been debugging the the functionality of the --for-status option because of the failure of test 6 of t7418-submodule-sparse-gitmodules.sh. Also, to my delight, my set-branch port is finally merged into master of git/git!

What is the test about?

The test t7418 mainly involves finding summaries of submodules in case the .gitmodules is missing. Such a scenario may be created because of sparse checking-out a repository and doing it in such a way that the .gitmodules is not listed in the sparse checkout’s cone. I learned about sparse checkout from this article by one of the GSoC mentors and contributor Derrick Stolee. I will not dive much into sparse checkout since it is out of scope of this article, but to give a gist of it in simple terms:

Performing a sparse checkout creates a “cone” of objects currently
checked out by Git. The cone is basically a set of entities
(files/directories/pathspecs) that we need as we may want to work
solely upon them. Mind you that a sparse checkout will NOT delete the
non-checked out objects but rather not perform various git actions
such as log, blame, etc. on them.

Therefore, in our case, the .gitmodules file is not checked out in the current cone.

What is for-status?

The for-status option will help us to fetch the summaries of submodule(s) whose .gitmodules is not currently available by looking at the .gitmodules file from any previous commits and then fetch the details. This is one of those things which gets me awestruck about how well Git does in scenarios in which doing operations seems next to impossible! Imagine finding out a summary even if the .gitmodules is missing! The option is used as follows:

git submodule summary --for-status <rev> <path>

Passing the revision is not compulsory and will be assumed as HEAD in absentia.

The problem

The problem is that the option is not working correctly; too vague of an answer isn’t it? Well to dive a bit deeper, this particular segment of prepare_submodule_summary() is not behaving well for some reason

if (info->for_status) {
char *config_key;
const char *ignore_config = "none";
const char *value;
const struct submodule *sub = submodule_from_path(the_repository,
		&null_oid, p->sm_path);

if (sub && p->status != 'A') {
	config_key = xstrfmt("submodule.%s.ignore",
			     sub->name);
	if (!git_config_get_string_const(config_key, &value))
		ignore_config = value;
	else if (sub->ignore)
		ignore_config = sub->ignore;

	free(config_key);
	if (!strcmp(ignore_config, "all"))
		continue;
	}
}

Now one may ask how I have narrowed out the error to this segment? The thing is that the part responsible for outputting the main header of the summary output i.e.,

* submodule sha1...sha2 (n):

does not actually output the (n) part in case n < 0 which means that the part responsible for finding out n or the total number of commits did not run due to some reason. After some more debugging and following these small leads I reach to the point mentioned above. This is the diff between the expected and actual outputs of test 6 of t7418:

+ diff -u expect actualmerged
--- expect      2020-06-27 19:44:17.404389608 +0000
+++ actual      2020-06-27 19:44:17.400389586 +0000
@@ -1,3 +1,2 @@
-* submodule 83d36b8...a939200 (1):
-  < Add file2 to submodule
+* submodule a939200...a939200:

As you may see, the (n) and the commit message are missing from the actual output. Therefore, my doubts are most probably correct.

My mentor Kaartic also reported some problems with the output of summary when he ran it on his lxconf submodule testing repository. This is the actual output:

$ git submodule summary 7c1532cab3125ae24f14ee9433b3c673f2964ef1
* bash 4f90705...f10c25c:

While the expected output was:

$ git submodule summary 7c1532cab3125ae24f14ee9433b3c673f2964ef1
* bash 4f90705...f10c25c (4):
  > Git: add config for hiding prefix shown in diff
  > tools: add cron script to automatically build,install git(-next)
  > tools: add script to check conflicts in sync(thing) folders
  > Git: use the experimental commit graph feature

Again as you may see that there is some problem with the tallying of the total number of commits. Therefore, even if my estimate of the inaccuracy of the aforementioned for-status if-segment may be wrong, this much can be said for certain that the commit tallying is not right.

One thing to note is that the outputs are perfect for t7401-submodule-summary.sh (the test actually designed to test summary). I confirmed this by doing a cat actual at various places and noticed the good outputs we got. Hence there is a chance even be a problem with the test too?

The solution?

Well I am working on it. I am betting on the wrong commit tallying hence I will have to explore where this is going wrong exactly. I dig down a bit deeper to see what exactly is up with the struct submodule (defined in submodule-config.h). There is a variable called struct object_id gitmodules_oid which most probably is the object holding the .gitmodules file’s information. To back my claims, I did a git log --format=%B -n 1 959b5455d07 (found this commit by doing a git blame on the struct) which revealed a commit log detailing the introduction of a configuration API to ease the look-up of .gitmodules values. I get a feeling that we may want to use this variable in the --for-status option. I am surprised this hasn’t been used anywhere in the if segment.

EDIT: On further debugging I found that the submodule’s head ref is not being recognised by Git for some reason and we get a ENOTDIR error when an lstat() is done on the submodule/.git/HEAD. Hence, another temporary fix I have tried is passing the URL of the submodule instead of the path to check if things work; surprisingly, atfer the fix t7418 works perfect while 19/23 tests fail in t7401 :(

Next steps

I have to figure out a solution to this quickly as we are approaching the first evaluation of GSoC in a couple of days. Again, there may be more to it than what meets the eye but I will have to place my bets somewhere or the other. I hope some good things are on their way!

Over and out,
Shourya Shukla

Location: Chennai, Tamil Nadu, India

Comments

Post a Comment

Popular posts from this blog

The Final Report

The Final Report This blog will describe my GSoC project along with a small account of how I started contributing to Git. My GSoC '20 project was to convert 'submodule' to a builtin by porting it from shell to C. Initially, Git commands were written in shell with some instances of Perl as well. As times progressed, various platforms to run Git emerged & projects became large (spanning millions of lines of code), enter, problems in production level code, such as: Difficulties in portability of code. The submodule shell script uses commands such as echo, grep, cd, test and printf to name a few. When switching to non-POSIX compliant systems, one will have to use emulation layers to implement such commands on the system. Which is a lot of extra work. There is large overhead involved in calling the command . As these commands implemented in shell script are not buitlins, they tend to call multiple fork() and exec() syscalls for creating more child p...
Read more

GSoC Week 4 [One month special]

 One month complete! A month has passed now since GSoC started (it will on June 4) and it is a fine milestone I think! I feel happy about it. This week was more about studying things and what is under the hood rather than programming something like it has been always. I was stuck with the callback mechanism of the submodule code. I could not understand the purpose behind it and thought that it was not needed. But as always, the things which confuse me are the ones with very simple and crystal clear logic behind them (but again, not to fret, we learn to walk by falling over and over right?); I will cover the callback mechanism in the blog so that anyone who ever wants to work on submodules or a future student like me has some reference to what is going on. What are callbacks? Callbacks are defined in regard to a function or a segment of a program you call when encountered with a particular trigger (such as an if statement or end of function). And by “you call”, I ...
Read more

GSoC Week 10

Addressing comments on my patch I spent this week addressing the suggestions made by Johannes which, were extremely detailed and long. I am done with most of the suggestions so I will highlight some important ones and what I learned from them. This week, I saw the ocean for I guess the first time in my life and it was a totally unique experience! Peaceful Seas Moving to the comments made by Johannes. Eliminate find_unique_abbrev() The shell version of summary calls git rev-parse twice for two jobs which are: Checking for a missing_src and a missing_dst i.e. , the missing source and destination committish. test $mod_src = 160000 && ! GIT_DIR="$name/.git" git rev-parse -q --verify $sha1_src^0 >/dev/null && missing_src=t test $mod_dst = 160000 && ! GIT_DIR="$name/.git" git rev-parse -q --verify $sha1_dst^0 >/dev/null && missing_dst=t Quoting my mentor Kaartic: The above code assigns missi...
Read more