Granularity and the Burnout. (1 Viewer)

Thales750

Formerly Jsanders
Local time
Today, 15:11
Joined
Dec 20, 2007
Messages
2,087
After spending a number of years in a small business with early stage growth clients, I've seen a lot.

Manager, and Developer, overreach is a very common source of high stress and failed projects. We all know about minimum deliverables we all know to reign in overreach, and yet, it plagues us.

When developing Mission Critical databases, how do you manage (plan) that fine line between fine granularity and impossible to complete?
 

The_Doc_Man

Immoderate Moderator
Staff member
Local time
Today, 14:11
Joined
Feb 28, 2001
Messages
27,146
Planning is imperative. There is also the need to keep the marketing and finance people out of engineering meetings (if that is applicable) or at least learn how to throttle them for planning sessions.

Once, perhaps 40 years ago, I was a department head and lead software designer for my employer. We had to bid on a job with lots of bucks but also a lot of strict requirements. I did my due diligence, scoped out the going rate at which we assembled software elements including integration and did my best on estimating the unique elements of programming that would NOT be off-the-shelf. Turned in my numbers. The marketing guy said, "We can't do that, we'll never get the job, your software costs too much." Long argument ensued and I told them that in a risk/reward analysis, the new hardware required for the job was very high risk because we had no device driver for it and would have to write one. Bounced off his back like water off a duck. He said, "Time to sharpen the pencil" and I told him I already had, and there was no going lower without incurring massive non-performance risk. Finally, the company president sided with me because he had heard the "risk" argument and understood it. (And I had a nice spreadsheet detailing what was easy and what wasn't.)

So we bid what we thought it would take. Another company underbid us. The marketing guy visited me with the bad news and said, "Thanks for losing us that job. Maybe next time you'll listen to me." Three weeks later, the underbidder defaulted and paid a stiff cash penalty. The story got back to us from the customer that they realized they would lose a lot of money at their price and could not meet the schedule. We got the job at our original price and they even relaxed the penalty clauses rather than lose another vendor. I found the marketing guy and simply raised my eyebrow. He slunk away without saying a word and never argued again on engineering risk.

How does that help you? A "Mission Critical" database involves lots of risks. A good designer recognizes which steps are easy and which ones are not at all easy. Your goal is to either minimize risk or to identify risks that cannot be avoided but that can be mitigated by allowing extra time / effort / money to get those parts right. The key to success in major projects is detailed planning which includes enough analysis to identify situations where you can lose your shirt. Then ask "How can I keep my shirt intact and still on my back?"

Your question about granularity vs. completion is directly related to setting realistic goals that include as much as is reasonably representative of the business model you are building. If you build the model but it is not granular enough to provide answers, what good is it? You need to think outside the box in this way... if your project is late, what happens? Well, how did your business run without the new project? Was that drop-dead date someone's pipe dream or was it that some expensive license expired or a loan became due or something else happened with some teeth to it? Prior detailed analysis is the sine qua non for project management.
 

Pat Hartman

Super Moderator
Staff member
Local time
Today, 15:11
Joined
Feb 19, 2002
Messages
43,231
I can't tell you how many times managers tried to intimidate me into lowering my bids. I even had one ask me if it would cost less if we did it in 5 modules instead of 7:) :) He just shook his head when I informed him that it would actually cost more because we would be creating pathological connections where none were necessary and testing would be much more complex.
 

The_Doc_Man

Immoderate Moderator
Staff member
Local time
Today, 14:11
Joined
Feb 28, 2001
Messages
27,146
With regard to project management, your typical progress chart connects tasks according to dependencies. By merging two modules into one, you introduce a connection for code that has nothing to do with the thing which precedes it in the dependency chart. In project management, this is one example of a pathological connection. Or at least that is how I remember it from 40+ years ago when I was a department manager and actually had to diddle with progress charts and topological sorting.

Say you are building modules to do text parsing and the boss says, "Include the routines for linear regression support in that parser module." Later in the schedule, one form depends on regression. Another form depends on text parsing. But now in that schedule, two forms depend on the same something that is only partly involved with either. If you had kept the regression and parsing routines in separate modules and allowed them to have their own independent start/end dates, the regression form would not have depended on parsing code, which it doesn't use anyway.

The problem, of course, is that when you create that double-barreled dependency, BOTH forms suddenly align to the same start time, and necessarily, one of those two forms now has a later start time than it should have. Therefore, collapsing the number of modules to contain the same number of function/sub entry points can introduce delays that shouldn't be present. And of course, splitting the dependencies means you should have kept the modules split, too. Because it is a project no-no to say "project point X now depends on half of a routine being finished."
 

isladogs

MVP / VIP
Local time
Today, 20:11
Joined
Jan 14, 2017
Messages
18,209
Whilst I'm grateful to Doc for responding, I'd still like to know what Pat was referring to as she used that phrase (which is new to me) twice recently.
 

Thales750

Formerly Jsanders
Local time
Today, 15:11
Joined
Dec 20, 2007
Messages
2,087
Whilst I'm grateful to Doc for responding, I'd still like to know what Pat was referring to as she used that phrase (which is new to me) twice recently.
Did you read Doc's answer?

I think it means beyond this point there be dragons.
We make connections that should never have been made. Then in the execution, a false dependency occurs.
 

The_Doc_Man

Immoderate Moderator
Staff member
Local time
Today, 14:11
Joined
Feb 28, 2001
Messages
27,146
To be fair, different places have different names for some issues. We actually called them pathological dependencies, not connections. But that sort of "clicked" when I read Pat's use thereof. If that WASN'T what she meant, I'll admit to some curiosity as well.
 

Thales750

Formerly Jsanders
Local time
Today, 15:11
Joined
Dec 20, 2007
Messages
2,087
I went back to school this morning Doc. I revisited Linear Regression. That led to a couple of tangents, which is predictable. Curiosity being X and all.
 

The_Doc_Man

Immoderate Moderator
Staff member
Local time
Today, 14:11
Joined
Feb 28, 2001
Messages
27,146
Just for snorts and giggles, I actually contributed an article to the forum some years ago on the subject of a "quick and dirty" linear regression and also contributed a class module for ASCII text parsing techniques. Which is why I picked those two disparate subjects.
 

gemma-the-husky

Super Moderator
Staff member
Local time
Today, 20:11
Joined
Sep 12, 2006
Messages
15,640
I think a lot of the problem comes from users trying to design an application based on what they think they need, ie their job functions rather than the management of the data, and maybe also the developers not quite understanding what the users really wanted, but didn't specify correctly.

(I hope this describes the issue under discussion)
 
Last edited:

isladogs

MVP / VIP
Local time
Today, 20:11
Joined
Jan 14, 2017
Messages
18,209

Isaac

Lifelong Learner
Local time
Today, 12:11
Joined
Mar 14, 2017
Messages
8,777
Get comfortable saying "No".
 

Pat Hartman

Super Moderator
Staff member
Local time
Today, 15:11
Joined
Feb 19, 2002
Messages
43,231
@isladogs Before there was Object Oriented Design/Programming there was Structured Design/Programming. Coupling and Cohesion are concepts that have been around since the 60's and form the basis of Structured Design/Programming and it is too bad they are no longer taught. Coupling refers to how tightly connected two processes are and Cohesion refers to how related processes internal to a module are. Pathological Coupling is a general term that implies two processes are too interconnected to be split and possibly shouldn't have been connected at all. We run into this frequently when modifying code. You change A and all of a sudden B doesn't work any more. It's like having your Accounts Payable break when someone changes Payroll because they both produce checks and someone hardcoded something that made them inter dependent so when you switched check vendors for Payroll, the format no longer worked for AR. The original connection was temporal because they both ran the first Tuesday of the month. Pathological connections are not generally part of the original design. They tend to occur during maintenance when someone takes a shortcut and reuses some code improperly.
 

isladogs

MVP / VIP
Local time
Today, 20:11
Joined
Jan 14, 2017
Messages
18,209
Many thanks Pat for your insight into this topic (and again to Doc)

I have to admit that I'd never previously heard the two words pathological coupling used together and if I'd been asked to guess what it meant would probably guessed it was the precursor to conscious uncoupling rather than anything to do with programming.

For more info, see Thoughts on Coupling in Software Design | Codurance
Content Coupling: Content coupling, or pathological coupling, occurs when one module modifies or relies on the internal workings of another module. Changing the inner working will lead to the need of changing the dependent module.

Trying to research this topic earlier reminded me of when I used to teach students how to do internet searches most successfully some 20 years ago. The lesson starter activity was usually a challenge for students to find the single word with the most hits.
That was used as a lead-in to techniques for making searches more effective.
The final activity was another challenge for students to find a googlewhack:

a googlewhack is a search term consisting of two words, with no surrounding quotation marks, that produces one single result when entered into the search engine Google.

Twenty years ago that was relatively easy and usually at least one student in each group found a suitable combination of words. I used to have many such examples. However as the game became more popular, many people published their results online resulting in there being at least two hits for that phrase once the search bots did their job.

Nevertheless, after a few minutes random searching, I found this googlewhack (but that was due to an unintentional misspelling of crapulent!)

1629802709218.png



NOTE Surprisingly, using Bing, it got lots of hits:

1629802826979.png


P.S. Apologies for hijacking this thread into a different direction
 

Attachments

  • 1629802662841.png
    1629802662841.png
    43.4 KB · Views: 165
Last edited:

The_Doc_Man

Immoderate Moderator
Staff member
Local time
Today, 14:11
Joined
Feb 28, 2001
Messages
27,146
Thanks for the clarification, Pat. My discussion was almost right. In the case I was describing, the pathology was caused by forcing improper cohesion that resulted in schedule-level coupling. Your definition is more general. Been a while since I dealt with Structured Programming issues. But then, I was always the maverick when it came to structured programming. For a long time I refused to work in any language that didn't have a GOTO. Had to break my own rules later because of SQL.
 

Pat Hartman

Super Moderator
Staff member
Local time
Today, 15:11
Joined
Feb 19, 2002
Messages
43,231
Once I saw the light, I never wrote another GoTo again UNTIL I started using CICS (on line transaction programming. This is what you saw on all those clunky CRT's that were as big as a house and were only 80 characters wide and 24 lines deep) because of the way CICS handled errors, you had NO OPTION but to use GoTo to escape from an error. For that reason I never liked CICS. I always preferred IMS/DC which provided the same type of transaction environment but was more rationally in my opinion. In fact, I refused to put exit paragraphs following all normal paragraphs in my COBOL programs (except for the CICS transactions). Therefor my code would be
Perform ClearVariables. --- since that would prevent people from adding GoTo's to my code whereas the other camp would use
Perform ClearVariables Through ClearVariablesExit. ---- That structure allowed internal code to "exit" by using GoTo ClearVariablesExit.

"Perform somename" is the equivalent of GoSub in VBA. It implies a return at the end of the procedure.

Both CICS and IMS/DC performed the functions that web pages perform today except they were not as pretty and all the users were connected via the network. There was no such thing as an anonymous user. When I was working for large insurance companies like The Hartford Insurance Group, I might have as many as 4,000 users across the country working with one of my simpler CICS Transactions at one time. Talk about pressure to optimize your code. Our service level agreements with the user community generally promised sub-second response for most inquires and the longest agreed response I ever saw was 4 seconds. But hey, we weren't sending graphics and dancing dog animations. we were simply sending 80x24 text.

It is kind of amazing what programming without GoTo's does to your though process and design. Your procedures became much smaller and more cohesive. Because you had only a single exit point, you couldn't mush a lot of code together that didn't really belong together which was the entire point after all:)
 

The_Doc_Man

Immoderate Moderator
Staff member
Local time
Today, 14:11
Joined
Feb 28, 2001
Messages
27,146
My only real complaint about "NO GOTO" is when I have a deeply nested set of decisions, any one of which had the potential to abort the whole nest yet not exit the subroutine where they existed. In our industrial environment such situations cropped up now and then. Rare, but happened. Usually occurs when an elegant solution doesn't present itself and you are left with brute force solutions.
 

Users who are viewing this thread

Top Bottom